[Lustre-discuss] Slow performance on the first IOR/iozone (Lustre)

[Andreas Dilger]

On Mar 25, 2008  17:43 +0100, C�dric Lambert wrote:
> Your remark is completely justified : theory can be limited by some 
> external variables.
> 
> For example, we saw there was a limit with "ext2" file system : 
> EXT2_MAX_GROUP_LOADED is used to prevent too much cache bitmaps into 
> memory.  As Lustre is based on ext3, is there such a limit with ldiskfs 
> ? If so, we could continue to have bad performance again with big big 
> files on all OSTs (for example).

This MAX_GROUP_LOADED parameter was removed, and was pointless in any
case because the block bitmaps were still kept by the buffer cache if
used frequently.

> I let Anand tell us if results are better on its benches after a 
> dumpe2fs on every device...

I'm not sure that dumpe2fs will be sufficient because the ldiskfs
mballoc code also generates buddy bitmaps for every group it is
doing allocations in.  Reading the block bitmaps into cache in advance
will definitely help this, but dumpe2fs will not trigger the buddy
bitmap generation.


> Brian J. Murrell a écrit :
> > On Tue, 2008-03-25 at 10:37 -0400, Oleg Drokin wrote:
> >   
> >>     You certainly can prefetch that data sooner as a workaround and  
> >> with zero kernel hacking
> >>     required by e.g. doing dumpe2fs on every device with lustre  
> >> backend fs after lustre servers
> >>     are started (it is important to do this after mount, since kernel  
> >> discards all block device data
> >>     on last device close).
> >>     
> >
> > Cédric,
> >
> > If nothing else, you could use this technique to confirm Oleg's theory
> > that it is indeed bitmaps being cached that is resulting in your
> > performance observations.  Not that I doubt Oleg's expertise here.  It
> > would just be scientific data to confirm your situation.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.