[Lustre-discuss] lustre and small files overhead lru_size question

Andreas Dilger adilger at sun.com
Thu Jun 12 13:19:30 PDT 2008


On Jun 12, 2008  12:42 -0700, Cliff White wrote:
> Jakob Goldbach wrote:
> >> I rather think of increasing lru_size only on few login nodes where  
> >> user access file system interactively.
> > 
> > I run with 20k on a few-node cluster without problems. I also increased
> > max_dirty_mb. Both improved small I/O.
> 
> Basically, you want the total ( lru_size summed for all clients )  to
> be less than ~500k. If you stay below that limit, should have no issues.

Exceeding this limit isn't necessarily going to cause problems either, it
is really a function of your RAM size on each server.  TACC Ranger has
64k cores, so it would have a maximum of 64000 * 100 = 6.4M locks on
each MDT/OST (16GB RAM or so?) with the default 100 locks/core on clients.
With multiple OSTs/OSS this number would increase correspondingly.

>  From a conversation a few years ago:
> 
> "
> If this is going to be a compile client instead of a compute client, the
> standard is to increase the MDC LRU size to 5000, and the OSC LRU size 
> to 2000.
>   If they don't have thousands of clients this could be done on a large 
> number of clients if this test represents their normal workload (aim for
> LRU * clients <= 500k should be safe)."

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.




More information about the lustre-discuss mailing list