[Lustre-discuss] bad 1.6.3 striped write performance

Robin Humble rjh+lustre at cita.utoronto.ca
Mon Nov 26 06:32:59 PST 2007


On Mon, Nov 26, 2007 at 02:53:25PM +0100, Johann Lombardi wrote:
>On Mon, Nov 26, 2007 at 08:39:58AM -0500, Robin Humble wrote:
>> Hi,
>> 
>> I'm seeing what can only be described as dismal striped write
>> performance from lustre 1.6.3 clients :-/
>> 1.6.2 and 1.6.1 clients are fine. 1.6.4rc3 clients (from cvs a couple
>> of days ago) are also terrible.
>> 
>> the below shows that the OS (centos4.5/5) or fabric (gigE/IB) or lustre
>> version on the servers doesn't matter - the problem is with the 1.6.3
>> and 1.6.4rc3 client kernels with striped writes (although un-striped
>> writes are a tad slower too).
>> 
>> with 1M lustre stripes:
>>    client              client                     dd write speed (MB/s)
>>      OS                kernel                     a)    b)    c)    d)
>> 1.6.2:
>>   centos4.5  2.6.9-55.0.2.EL_lustre.1.6.2smp     202   270   118   117
>>   centos5    2.6.18-8.1.8.el5_lustre.1.6.2rjh    166   190   117   119
>> 1.6.3+:
>>   centos4.5  2.6.9-55.0.9.EL_lustre.1.6.3smp      32     9    30     9
>>   centos5    2.6.18-53.el5-lustre1.6.4rc3rjh      36    10    27    10
>>                                                        ^^^^        ^^^^
>>                                          yes, that is really 9MB/s. sigh
>
>Could you please try to disable checksums?
>On the client side:
>for file in /proc/fs/lustre/osc/*/checksums; do echo 0 > $file; done

done. no change.

cheers,
robin




More information about the lustre-discuss mailing list