[Lustre-discuss] small file performance

[Cliff White]

Aaron Knister wrote:
> Striping is turned off. Are there any other optimizations you know of  
> to increase the speed of metadata operations?

Not metadata specific, but increasing
/proc/fs/lustre/ldlm/namespaces/*/lru_size on the affected clients
will help in some cases. This parameter controls locks cached on the
client. Increasing it should help your compiles.

This parameter can be set per-client, you do not have to alter server-side.

cliffw

> 
> On Jan 5, 2008, at 3:50 AM, Robin Humble wrote:
> 
>> On Fri, Jan 04, 2008 at 09:44:54AM -0500, Aaron Knister wrote:
>>> For whatever reason, searching my lustre mount (ls -R or find),
>>> compiling code and other operations involving lots of small files are
>>> painfully slow. There is no load on the filesystem other than my
>>> various tests. I've disabled lnet debugging. Just to give you an idea
>>> of how slow it is-- a ./configure of this particular code on a local
>>> filesystem takes less than a minute. On lustre it's been running for
>>> five minutes and is hardly half way through. An untar on the local
>>> filesystem takes .9 seconds while that same untar takes 12 seconds to
>>> our lustre mount. Any ideas for improving this?
>> do you have striping turned off?
>> that makes a massive difference for metadata operations...
>>  lfs setstripe -d /some/lustre/dir/
>>
>> cheers,
>> robin
> 
> Aaron Knister
> Associate Systems Analyst
> Center for Ocean-Land-Atmosphere Studies
> 
> (301) 595-7000
> aaron at iges.org
> 
> 
> 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss