[Lustre-discuss] Solid State MDT

[Oleg Drokin]

Hello!

On Apr 13, 2009, at 12:55 PM, Jim Garlick wrote:
> 2) some quick tests of MDS create rates (through lustre now) on the  
> SSD
> and DDN hardware where we seemed to get about 2350 creates/sec no  
> matter
> what hardware we used, and posts from Oleg on this mailing list  
> indicating
> that tests utilizing loopback devices were only getting about 5300
> creates/sec:

Please note that 5300 that I got was for a single client! (btw, I hope  
you did not
use -y option to mdtest)
Since that time I had a chance to perform multi-client tests at ORNL  
(not ssd based,
but this is unimportant since we turned out to be cpu bound at a  
certain point anyway).
The result was 18k creates/sec for mkdirs (for some sort of 16 way  
fast cpus) (needs
patch from bug 18534 to reduce unneeded rpcs during create).
Actual open-creates would be slower right now to around 10k-12k  
creates/sec I would
estimate (assuming your OSTs can keep up with creationg, we do some  
investigations in
this area and also have found some problems in the mds-precreate- 
reqesting code already).

> http://lists.lustre.org/pipermail/lustre-devel/2009-February/002940.html
> So we're going with the 3ware setup for newer file systems for now
> and keeping the SSD config in our back pocket for further  
> investigation.

When approaching 18k creates/sec (total) (first with 8 clients), in  
initial test there was
a big dive at 16 clients that turned out to be journal overflow and so  
the syncing
slowed everything. This should not be a concern for SSDs, though.
Since we did not have an ssd in our back pocket at the time, we just  
tried
2G ramdisk-based journal instead, and that allowed us to remain at 18k  
creates/sec
plateau scaling creating from 8 to 32 clients doing creates, at which  
point we seem to
be overflowing the journal again (i know this is counterintuitive  
given how
the rate is the same and journal just got bigger).

Bye,
     Oleg