[Lustre-discuss] diskless booting & Debian

[Troy Benjegerdes]

> We've actually experimented a bit with this, since most of the nodes in
> our system are disk-less.  In fact, they're -less just about everything
> except for processors, memory, and a connection to our internal fabric. 
> We've tried loading "canned" MDT/OST images into memory on some nodes
> and serving from there, and it does seem to work.  There are two
> downsides, though.  One is that the Linux loopback driver is a real
> performance bottleneck, ever since some bright person had the idea to
> make it less multi-threaded than it had been.  Another is that booting
> tends to involve metadata-heavy access patterns which are not exactly
> Lustre's strength - a situation made worse when you have nearly a
> thousand clients doing it at the same time and your MDS is a relatively
> small node like the others.  So far we've found that NBD serves us
> better in the boot/root filesystem role, though that means a read-only
> root which involves its own complexity.  Your mileage will almost
> certainly vary.

Does the metadata updates problem go away in a read-only root
environment? That makes life a lot easier. I don't want to have some
random node that's either hacked or out to lunch be able to make changes to
the filesystem.

When I used NFS root, we had one node that was allowed to write, and all
the rest were read-only mounts. I find AFS much more convenient since I
can log in as root, and authenticate to the filesystem from any node to
make changes.