[Lustre-discuss] stripe offset and hot-spots

Peter Grandi pg_lus at lus.for.sabi.co.UK
Fri Nov 27 08:45:21 PST 2009


[ ... ]

jwhite> In reality, we have a quite varied workload on our file
jwhite> systems with codes ranging from bio to astrophys and, as
jwhite> such, writes ranging from very small to very large.

Good luck with that -- there is no file system design that can cope
*well* with all of that. Of course every vendor will tell you
otherwise abutn their product :-).

Also, there is no filesystem (as in -- file system instance,
actual storage pool) that can cope well with all that either.

jwhite> Any real-world experience with these situations?  Are
jwhite> there strange inefficiencies or administrative
jwhite> difficulties that should be known [ ... ]

Well, following on from the previous point, my recommendation
would be to consider using different tools for different purposes,
or at the very least different (storage) pools for different
purposes.

Lustre is a good candidate for the "silver bullet" sell because it
is indeed pretty decent all-round, and there isn't much else
(perhaps GlusterFS) in the same class, as NFS and SMB and OpenAFS
all have some greater limitations (especially for Linux).

Also it has an underappreciated aspect: it is pretty decent even as
a non-cluster (e.g. one MDT/some OSTs a single server) filesystem,
because it is based on something close to 'ext4', which is ok-ish,
and because its network protocol is fairly ok and with arguably a
better design than the NFS or SMB protocol and quite importantly a
better client implementation than the Linux NFS or SMB clients.

But even if using Lustre for many sorts of things, I would not use
a single giant storage pool anyhow.

Sure "management" consultants push the sweet drug of consolidation
and virtualization, but taken to an extreme it creates a lot of
problems, as a single immense storage pools becomes rigid and hard
to manage.

However in many cases "management" can be easily sold on the
"silver bullet" theory that a single tool and pool can support
every requirement and the so called "reality principle" need not
apply.

If your case is not like that one possible change in your situation
is to use Lustre for a lot of different applications, but have
several different storage pools with rather different storage
backends and tuning.



More information about the lustre-discuss mailing list