[Lustre-discuss] Large directory performance

Bernd Schubert bs_lists at aakef.fastmail.fm
Fri Sep 10 16:32:35 PDT 2010

On Saturday, September 11, 2010, Andreas Dilger wrote:
> On 2010-09-10, at 12:11, Michael Robbert wrote:
> > Create performance is a flat line of ~150 files/sec across the board.
> > Delete performance is all over the place, but no higher than 3,000
> > files/sec... Then yesterday I was browsing the Lustre Operations Manual
> > and found section 33.8 that says Lustre is tested with directories as
> > large as 10 million files in a single directory and still get lookups at
> > a rate of 5,000 files/sec. That leaves me wondering 2 things. How can we
> > get 5,000 files/sec for anything and why is our performance dropping off
> > so suddenly at after 20k files?
> > 
> > Here is our setup:
> > All IO servers are Dell PowerEdge 2950s. 2 8-core sockets with X5355  @
> > 2.66GHz and 16Gb of RAM. The data is on DDN S2A 9550s with 8+2 RAID
> > configuration connected directly with 4Gb Fibre channel.
> Are you using the DDN 9550s for the MDT?  That would be a bad
> configuration, because they can only be configured with RAID-6, and would
> explain why you are seeing such bad performance.  For the MDT you always

Unfortunately, we failed to copy the scratch MDT in a reasonable time so far. 
Copying several hundreds of million files turned out to take ages ;) But I 
guess Mike did the benchmarks for the other filesystem with an EF3010.

> > We have as many as 1.4 million files in a single directory and we now
> > have half a billion files that we need to deal with in one way or
> > another.

Mike, is there a chance you can try which rate acp reports?


Also could you please send me your exact bonnie line or script? We could try 
to reproduce it on and idle test 9550 with a 6620 for metada (the 6620 is 
slower for that than the ef3010).


Bernd Schubert
DataDirect Networks

More information about the lustre-discuss mailing list