[Lustre-discuss] Small files performance

Peter Grandi pg_lus at lus.for.sabi.co.UK
Fri Oct 30 11:05:08 PDT 2009


[ ... ]

> - will writing files that are smaller than the stripe_size and
>   that are written to a fully striped directory create objects
>   on more than 1 OST?

Let's say that in some previous discussion one of the Sun/CFS
people mentioned storing collections of small files into 'ext2'
image files on Lustre and then loopback mounting those on
clients.

While I have mentioned using DB4 or similar indexed databases.

> - what would be the main options to tune in order get the best
>   out of Lustre for small files? [ ... ]

That depends a lot on the specific access patterns, and what you
mean by "small".

I guess, also from reading some previous comments on the topic,
that the first thing to consider is appropriate storage backends
(e.g. SSD, and Sun/CFS otherwise recommend RAID10) for the MDTs,
and maybe even for the OSTs. Lots of small files, lots of IOPs
(usually).



More information about the lustre-discuss mailing list