[Lustre-discuss] Performance Drop creating big files

[Lundgren, Andrew]

Are you suggesting he use machines with 40G of RAM to work with 10G files?

We have many 800-900G files...  I am not sure that is a realistic number.

--
Andrew

> -----Original Message-----
> From: lustre-discuss-bounces at lists.lustre.org
> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of
> Kumaran Rajaram
> Sent: Friday, May 30, 2008 3:01 PM
> To: Roger Spellman
> Cc: lustre-discuss at lists.lustre.org
> Subject: Re: [Lustre-discuss] Performance Drop creating big files
>
> Roger,
>
> I would suspect the Lustre client side caching influencing your write
> performance. 10GB is not adequate, try atleast 4x of the RAM size for
> file-sizes. Try doing the same tests with O_DIRECT flag as it'd truly
> measure your disk I/O performance bypassing FS + buffer cache.
>
> HTH,
> -Kums
>
> On Fri, 2008-05-30 at 16:25 -0400, Roger Spellman wrote:
> > I am seeing the following odd behavior.  I have several OSSes, each
> > with a 7T RAID 5.
> >
> >
> >
> > If I use a single client to create a single 1T file which is striped
> > to a single OST, the performance starts off at about 400 MB/s (which
> > is typical for my HW), then gradually decreases, until it
> reaches 250
> > MB/s.  I've seen this with both IOZone and dd.
> >
> >
> > As an experiment, I wrote a script that creates two hundred
> 10G files,
> > without removing them.  Again, the performance starts off
> at 400 MB/s.
> > But, the performance stays nearly the same throughout the test.
> >
> >
> >
> > The only difference between these tests is that in the second case,
> > there are lots of opens & closes, and in the first case,
> just a single
> > open and close.
> >
> >
> >
> > Can anyone explain what is happening here, and how to
> possible fix it?
> >
> >
> >
> > Thanks.
> >
> >
> >
> > Roger Spellman
> >
> > Sr. Staff Engineer
> >
> > Terascala, Inc.
> >
> >
> >
> >
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-discuss
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>