[lustre-discuss] Lustre poor performance

[Riccardo Veraldi]

thank you Keith,
I will do all this. the single thread dd tests shows 1GB/sec. I will do
the other tests


On 8/18/17 9:05 AM, Mannthey, Keith wrote:
> I would suggest you a few other tests to help isolate where the issue might be.  
>
> 1. What is the single thread "DD" write speed?
>  
> 2. Lnet_selfttest:  Please see " Chapter 28. Testing Lustre Network Performance (LNet Self-Test)" in the Lustre manual if this is a new test for you. 
> This will help show how much Lnet bandwith you have from your single client.  There are tunable in the lnet later that can affect things.  Which QRD HCA are you using?
>
> 3. OBDFilter_survey :  Please see " 29.3. Testing OST Performance (obdfilter-survey)" in the Lustre manual.  This test will help demonstrate what the backed NVMe/ZFS setup can do at the OBD layer in Lustre.  
>
> Thanks,
>  Keith 
> -----Original Message-----
> From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Riccardo Veraldi
> Sent: Thursday, August 17, 2017 10:48 PM
> To: Dennis Nelson <dnelson at ddn.com>; lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] Lustre poor performance
>
> this is my lustre.conf
>
> [drp-tst-ffb01:~]$ cat /etc/modprobe.d/lustre.conf options lnet networks=o2ib5(ib0),tcp5(enp1s0f0)
>
> data transfer is over infiniband
>
> ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 65520
>         inet 172.21.52.83  netmask 255.255.252.0  broadcast 172.21.55.255
>
>
> On 8/17/17 10:45 PM, Riccardo Veraldi wrote:
>> On 8/17/17 9:22 PM, Dennis Nelson wrote:
>>> It appears that you are running iozone on a single client?  What kind of network is tcp5?  Have you looked at the network to make sure it is not the bottleneck?
>>>
>> yes the data transfer is on ib0 interface and I did a memory to memory 
>> test through InfiniBand QDR  resulting in 3.7GB/sec.
>> tcp is used to connect to the MDS. It is tcp5 to differentiate it from 
>> my other many Lustre clusters. I could have called it tcp but it does 
>> not make any difference performance wise.
>> I ran the test from one single node yes, I ran the same test also 
>> locally on a zpool identical to the one on the Lustre OSS.
>>  Ihave 4 identical servers each of them with the aame nvme disks:
>>
>> server1: OSS - OST1 Lustre/ZFS  raidz1
>>
>> server2: OSS - OST2 Lustre/ZFS  raidz1
>>
>> server3: local ZFS raidz1
>>
>> server4: Lustre client
>>
>>
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>