[Lustre-discuss] Performance Drop creating big files

[Marty Barnaby]

To refer to your original discussion, how were you writing your 1TB 
file? I'm mainly wondering about the blocksize of the individual writes.

Marty Barnaby


Lundgren, Andrew wrote:
> I get it.
>
> Thanks!
>
>   
>> -----Original Message-----
>> From: Kumaran Rajaram [mailto:krajaram at sgi.com]
>> Sent: Friday, May 30, 2008 3:07 PM
>> To: Lundgren, Andrew
>> Cc: Roger Spellman; lustre-discuss at lists.lustre.org
>> Subject: RE: [Lustre-discuss] Performance Drop creating big files
>>
>>
>> I meant use file-size that are 4x the RAM size :-). So if
>> your RAM size
>> is 4GB, use atleast 16GB file-sizes for the benchmarks.
>>
>> On Fri, 2008-05-30 at 15:03 -0600, Lundgren, Andrew wrote:
>>     
>>> Are you suggesting he use machines with 40G of RAM to work
>>>       
>> with 10G files?
>>     
>>> We have many 800-900G files...  I am not sure that is a
>>>       
>> realistic number.
>>     
>>> --
>>> Andrew
>>>
>>>       
>>>> -----Original Message-----
>>>> From: lustre-discuss-bounces at lists.lustre.org
>>>> [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of
>>>> Kumaran Rajaram
>>>> Sent: Friday, May 30, 2008 3:01 PM
>>>> To: Roger Spellman
>>>> Cc: lustre-discuss at lists.lustre.org
>>>> Subject: Re: [Lustre-discuss] Performance Drop creating big files
>>>>
>>>> Roger,
>>>>
>>>> I would suspect the Lustre client side caching
>>>>         
>> influencing your write
>>     
>>>> performance. 10GB is not adequate, try atleast 4x of the
>>>>         
>> RAM size for
>>     
>>>> file-sizes. Try doing the same tests with O_DIRECT flag
>>>>         
>> as it'd truly
>>     
>>>> measure your disk I/O performance bypassing FS + buffer cache.
>>>>
>>>> HTH,
>>>> -Kums
>>>>
>>>> On Fri, 2008-05-30 at 16:25 -0400, Roger Spellman wrote:
>>>>         
>>>>> I am seeing the following odd behavior.  I have several
>>>>>           
>> OSSes, each
>>     
>>>>> with a 7T RAID 5.
>>>>>
>>>>>
>>>>>
>>>>> If I use a single client to create a single 1T file
>>>>>           
>> which is striped
>>     
>>>>> to a single OST, the performance starts off at about
>>>>>           
>> 400 MB/s (which
>>     
>>>>> is typical for my HW), then gradually decreases, until it
>>>>>           
>>>> reaches 250
>>>>         
>>>>> MB/s.  I've seen this with both IOZone and dd.
>>>>>
>>>>>
>>>>> As an experiment, I wrote a script that creates two hundred
>>>>>           
>>>> 10G files,
>>>>         
>>>>> without removing them.  Again, the performance starts off
>>>>>           
>>>> at 400 MB/s.
>>>>         
>>>>> But, the performance stays nearly the same throughout the test.
>>>>>
>>>>>
>>>>>
>>>>> The only difference between these tests is that in the
>>>>>           
>> second case,
>>     
>>>>> there are lots of opens & closes, and in the first case,
>>>>>           
>>>> just a single
>>>>         
>>>>> open and close.
>>>>>
>>>>>
>>>>>
>>>>> Can anyone explain what is happening here, and how to
>>>>>           
>>>> possible fix it?
>>>>         
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>>
>>>>> Roger Spellman
>>>>>
>>>>> Sr. Staff Engineer
>>>>>
>>>>> Terascala, Inc.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Lustre-discuss mailing list
>>>>> Lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>>>           
>>>> _______________________________________________
>>>> Lustre-discuss mailing list
>>>> Lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>>
>>>>         
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080530/6f849482/attachment.htm>