[Lustre-discuss] Fwd: I/O tests on Lustre

Marty Barnaby mlbarna at sandia.gov
Thu May 15 09:25:04 PDT 2008


I concur with Andreas; this question comes around regularly, but the all 
the details of the Lustre FS deployment, as well as host cluster 
configuration and implementation, have inordinate significance.

You can refer to the thread "Off-topic: largest existing Lustre file 
system?" back in January of this year. With the sole intention of 
finding a very high maximal rate, I ran benchmark MPI executions with 
exceptional parameters and got up to 28 GB/s. However, when I moderated 
any dynamic, the byte-rate performance often fell off quickly.

In general, your numbers seem reasonable to me.

 
Marty


Andreas Dilger wrote:
> On May 14, 2008  16:14 -0400, Garland Jackson wrote:
>   
>> I have a user at the Naval Research Laboratory running Lustre on a Cray
>> XD1.
>> I'll let his email to me tell the story, but he doesn't want to know if the
>> numbers he's observing are reasonable.
>>
>> What do you say out there?  Can anyone out there tell me if his numbers
>> make sense?
>>     
>
> It's completely dependent upon the hardware configuration (OSTs, disks,
> etc).  Some customers have 50GB/s filesystems, while I have a 40MB/s
> Lustre filesystem at home.
>
>   
>> Begin forwarded message:
>>
>>     
>>> From:
>>> Date: April 17, 2008 1:27:11 PM EDT
>>> To:
>>> Cc:
>>> Subject: I/O tests on Lustre
>>>
>>>
>>>   I ran the io test that I ran on the XD1 Lustre on the ice machine. This
>>> program writes data from
>>>   an increasing number of processors ( 1,2,4,8,16,32,64)  using MPI for
>>> parallelization and then
>>>   reads it back.  Enough data is written so that when I read the data has
>>> not been cached somewhere.
>>>   Data block sizes are large so my programs should be giving me peak
>>> bandwidths.
>>>
>>>   My last set of results were
>>>   Numbers of Processes     Write Rate   Read Rate
>>>                                            MBs/sec     MBs/sec
>>>                 1                          763.7       506.4
>>>                 2                        1248.8       968.7
>>>                 4                        1298.7       782.3
>>>                 8                        1292.5       900.6
>>>               16                        1298.1       820.7
>>>               32                        1296.7       809.3
>>>               64                        1296.3       878.9
>>>
>>>
>>>   The thirty two process failed with a time out but everything else worked
>>> ok.
>>>
>>>   My question is Do these rates look reasonable based on your Lustre
>>> experience?
>>>
>>>   Also any ideas on why I might be timing out on an occasional run?
>>>
>>>               wendell
>>>
>>>
>>>       
>
>   
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>     
>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Sr. Staff Engineer, Lustre Group
> Sun Microsystems of Canada, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20080515/a62cdf1e/attachment.htm>


More information about the lustre-discuss mailing list