[Lustre-discuss] Large directories optimization

[Andreas Dilger]

On Sep 17, 2009  13:28 +0200, Lukas Hejtmanek wrote:
> is it possible to optimize Lustre so that is supports really large directories
> (with 30k small files in it)? We have 8 physical clients which process jpeg
> files stored on Lustre volume and I get sooner or later client freezes - ls in
> Lustre directory waits forever. I there something I could do to improve
> performance?

We regularly test directories with 1M files in them, so I don't consider
30k files to be a large directory.  If you are always using small files
there are different things you can do to optimize performance, such as
using RAID-1+0 instead of RAID-5/6 on the OSTs.

> The lustre server is Build Version:
> 1.8.0-19700101010000-PRISTINE-.usr.src.lustre-prod.linux-2.6.22.19-2.6.22.19
> 
> The lustre client is Build Version:
> 1.6.7.1-19700101010000-PRISTINE-.scratch.xhejtman.suse-2.6.22.17-0.1-2.6.22.17-0.1-xen-lustre

Using a more common kernel (e.g. RHEL5.3) means a lot more people are
testing the same code as you are.

> Lustre: stable-MDT0000-mdc-ffff8802855b7800: Connection to service
> stable-MDT0000 via nid x.x.x.x at tcp was lost; in progress operations using
> this service will wait for recovery to complete.

As other's mentioned, this seems like a network or server problem.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.