[lustre-discuss] Stripe size for osts

[Pawel Dziekonski]

On pon, 21 mar 2016 at 09:24:02 +0000, Dilger, Andreas wrote:
> On 2016/03/18, 12:52, "Kurt Strosahl" <strosahl at jlab.org> wrote:
> >Good Afternoon,
> >
> >    I'm experimenting with ost configurations geared more towards small
> >files and operations on those small files (like source code, and
> >compiling), and I was wondering about changing the stripe size so that
> >small files fit more efficiently on an ost.  I believe that would be the
> >--param lov.stripesize=XX option for mkfs.lustre, is that correct?  And
> >is there a lower limit that I should know about?
> 
> Just to clarify, the stripe size for Lustre is not a property of the OST,
> but rather a property of each file.  The OST itself allocates space
> internally as it sees fit.  For ldiskfs space allocation is done in units
> of 4KB blocks managed in extents, while ZFS has variable block sizes (512
> bytes up to 1MB or more, but only one block size per file) managed in a
> tree.  In both cases, if a file is sparse then no blocks are allocated for
> the holes in the file.
> 
> As for the minimum stripe size, this is 64KB, since it isn't possible to
> have a stripe size below the PAGE_SIZE on the client, and some
> architectures (e.g. IA64, PowerPC, Alpha) allowed 64KB PAGE_SIZE.
> 
> For small files, the stripe_size parameter is virtually meaningless, since
> the data will never exceed a single stripe in size.  What is much more
> important is to use a stripe_count=1, so that the client doesn't have to
> query multiple OSTs to determine the file size, timestamps, and other
> attributes.

Andreas,

default stripe size is 1MB. Is there a reason for that?
P



-- 
Pawel Dziekonski <pawel.dziekonski at wcss.pl>
Wroclaw Centre for Networking & Supercomputing, HPC Department
phone: +48 71 320 37 39, fax: +48 71 322 57 97, http://www.wcss.pl