Stateless Arch
 

Has anyone done any research on stateless ArchLinux instances.

A stateless Arch would be one where the root filesystem is mounted
read-only and nothing changes there. Thus it can mounted over network
(using NFS, NBD and similar) by several, diskless, PCs at the same time.


I plan to have per user HOME directories on a server (again NFS or
similar), and users credentials in LDAP.


/var/run beeing a link to a tmpfs /run, and by using systemd-journal
without /var/log/journal (it will store logs in memory) a lot of things
avoid hitting the disk already.


I'd use connman for handling the net connection and it seems to require
a writable /var/lib/connman/



Anyone with any experience with this?


--
дамјан

Stateless Arch
 

On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:

> I'd use connman for handling the net connection and it seems to require a
> writable /var/lib/connman/
>

No experiece at all, but I'd say that /var must be writeable too. Think of
it some like the /home of the system, so you should have one per machine in
the NFS server.

Just my two cents...

--
Rodrigo

Stateless Arch
 

On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:
> Has anyone done any research on stateless ArchLinux instances.
>
> A stateless Arch would be one where the root filesystem is mounted read-only
> and nothing changes there. Thus it can mounted over network (using NFS, NBD
> and similar) by several, diskless, PCs at the same time.
>
> I plan to have per user HOME directories on a server (again NFS or similar),
> and users credentials in LDAP.
>
> /var/run beeing a link to a tmpfs /run, and by using systemd-journal without
> /var/log/journal (it will store logs in memory) a lot of things avoid
> hitting the disk already.
>
> I'd use connman for handling the net connection and it seems to require a
> writable /var/lib/connman/
>
>
> Anyone with any experience with this?

I have been working towards initscripts allowing this, and various
upstreams (such as util-linux) should also support this setup. That
said, I have not actually tested this to any great extent, so don't
know how well it will work (feedback very welcome!).

What should work (but might not!): /etc and /usr (and /lib, /sbin,
/bin) should be able to be mounted read-only. I expect you'll have to
figure out how to deal with /etc/resolv.conf, I wonder if
NetworkManager has learnt how to deal with this gracefully since I
last checked...

What will not work: as Rodrigo said, you'll still need /var to be
mounted read-write, the point of /var is for applications to be able
to write to it. Moreover, /var must be unique to each installation,
and cannot be shared (you can put it on an NFS share though, just make
sure you have one for each machine). Moreover, even if /etc/ is
mounted read-only, you probably want one per machine. You might get
away with sharing it, but then all your hostnames will be the same for
instance. Importantly: you don't want /etc/machine-id to be shared by
different machines (as it needs to be unique). If you do decide to
share /etc, you can replace /etc/machine-id by an empty file and
systemd will create a random one at every boot (in /run) and use that
instead, so you should be fine in this respect.

HTH,

Tom

Stateless Arch
 

On Mon, Jul 9, 2012 at 10:51 AM, Tom Gundersen <teg@jklm.no> wrote:
> On Mon, Jul 9, 2012 at 10:10 AM, Damjan <gdamjan@gmail.com> wrote:
>> Has anyone done any research on stateless ArchLinux instances.
>>
>> A stateless Arch would be one where the root filesystem is mounted read-only
>> and nothing changes there. Thus it can mounted over network (using NFS, NBD
>> and similar) by several, diskless, PCs at the same time.
>>
>> I plan to have per user HOME directories on a server (again NFS or similar),
>> and users credentials in LDAP.
>>
>> /var/run beeing a link to a tmpfs /run, and by using systemd-journal without
>> /var/log/journal (it will store logs in memory) a lot of things avoid
>> hitting the disk already.
>>
>> I'd use connman for handling the net connection and it seems to require a
>> writable /var/lib/connman/
>>
>>
>> Anyone with any experience with this?
>
> I have been working towards initscripts allowing this, and various
> upstreams (such as util-linux) should also support this setup. That
> said, I have not actually tested this to any great extent, so don't
> know how well it will work (feedback very welcome!).
>
> What should work (but might not!): /etc and /usr (and /lib, /sbin,
> /bin) should be able to be mounted read-only. I expect you'll have to
> figure out how to deal with /etc/resolv.conf, I wonder if
> NetworkManager has learnt how to deal with this gracefully since I
> last checked...
>
> What will not work: as Rodrigo said, you'll still need /var to be
> mounted read-write, the point of /var is for applications to be able
> to write to it. Moreover, /var must be unique to each installation,
> and cannot be shared (you can put it on an NFS share though, just make
> sure you have one for each machine). Moreover, even if /etc/ is
> mounted read-only, you probably want one per machine. You might get
> away with sharing it, but then all your hostnames will be the same for
> instance. Importantly: you don't want /etc/machine-id to be shared by
> different machines (as it needs to be unique). If you do decide to
> share /etc, you can replace /etc/machine-id by an empty file and
> systemd will create a random one at every boot (in /run) and use that
> instead, so you should be fine in this respect.
>
> HTH,
>
> Tom

Reference for the machine-id stuff:
<http://0pointer.de/public/systemd-man/machine-id.html> or
machine-id(5) if you have access to a systemd system.

-t

Stateless Arch
 

On Monday 09 Jul 2012 10:10:07 Damjan wrote:
> Has anyone done any research on stateless ArchLinux instances.
>
> A stateless Arch would be one where the root filesystem is mounted
> read-only and nothing changes there. Thus it can mounted over network
> (using NFS, NBD and similar) by several, diskless, PCs at the same time.
>
> I plan to have per user HOME directories on a server (again NFS or
> similar), and users credentials in LDAP.
>
> /var/run beeing a link to a tmpfs /run, and by using systemd-journal
> without /var/log/journal (it will store logs in memory) a lot of things
> avoid hitting the disk already.
>
> I'd use connman for handling the net connection and it seems to require
> a writable /var/lib/connman/
>
>
> Anyone with any experience with this?

I run an ArchLinux processing cluster for work. The nodes are diskless, so I
can certainly offer some advice for getting this setup to work.

Most important tip: don't try to use NFSv4. You can't boot Arch over NFSv4
yet.

My setup has the nodes mounting root rw, but in practice they never touch it
except for when I run an upgrade or do some manual configuration, which I
usually do from a node (because it's easier).

Each node has a separate /var directory stored on the server. I added a
script that uses the initscripts hooks to mount the correct /var for each node
based on its hostname (which is assigned via DHCP). If you don't care about
retaining state, you could probably mount /var as a tmpfs and copy a template
into it, which would avoid needing to create a directory on the server for
each node.

I'd be happy to provide further details if you'd like some pointers, but I got
most of the important information from the Wiki, so pretty much all you need
is there.

Paul

Stateless Arch
 

> > I'd use connman for handling the net connection and it seems to require a
> > writable /var/lib/connman/
> >
>
> No experiece at all, but I'd say that /var must be writeable too. Think of
> it some like the /home of the system, so you should have one per machine in
> the NFS server.
>
> Just my two cents...

And /tmp.

Obviously you need to remount,rw to update and you may have problems
forcing it to remount,ro without a reboot. You will probably be
rebooting after running pacman, most of the time though anyway.

Much simpler on OpenBSD, mount -urf works well and every time and
there are far less reboots required too.


A grsecurity kernel can send you logs whenever a remount takes place and
can't miss any like when using monit or something.

A benefit over OpenBSD.

--
__________________________________________________ ______

Why not do something good every day and install BOINC.
__________________________________________________ ______

Stateless Arch
 

On 09.07.2012 10:10, Damjan wrote:
> Has anyone done any research on stateless ArchLinux instances.
>
> A stateless Arch would be one where the root filesystem is mounted
> read-only and nothing changes there. Thus it can mounted over network
> (using NFS, NBD and similar) by several, diskless, PCs at the same time.

Thomas told me that he set up an initramfs that allows to fetch the
rootfs using rsync. That way you save quite some bandwidth, gain speed
and you can keep the "old" system if you don't want newer software. You
also won't have to worry about breaking software running on a client
when updating the base system on the server (changes in config
files/libraries that are (re)loaded during runtime, resource files, ...).

This will however require (albeit small) disks in the clients.

--
Florian Pritz

Stateless Arch
 

On Monday 09 Jul 2012 10:11:43 Kevin Chadwick wrote:
> > No experiece at all, but I'd say that /var must be writeable too. Think of
> > it some like the /home of the system, so you should have one per machine
> > in
> > the NFS server.
> And /tmp.

/tmp is a tmpfs for a default Arch install, so you don't need to worry about
that.

Paul

Stateless Arch
 

On Monday 09 Jul 2012 10:08:51 you wrote:
> My setup has the nodes mounting root rw, but in practice they never touch it
> except for when I run an upgrade or do some manual configuration, which I
> usually do from a node (because it's easier).

Oh, also, my setup has a separate root shared by the nodes; they don't share
the same root as the server. I don't think there's a good way to avoid that,
at least not for my usecase.

Paul

Stateless Arch
 

> /tmp is a tmpfs for a default Arch install, so you don't need to worry about
> that.

So it is, likely on the clients too and so may not have applied
here anyway.

--
__________________________________________________ ______

Why not do something good every day and install BOINC.
__________________________________________________ ______