[lustre-discuss] Lustre poor performance

Mannthey, Keith keith.mannthey at intel.com
Fri Aug 18 09:05:20 PDT 2017


I would suggest you a few other tests to help isolate where the issue might be.  

1. What is the single thread "DD" write speed?
 
2. Lnet_selfttest:  Please see " Chapter 28. Testing Lustre Network Performance (LNet Self-Test)" in the Lustre manual if this is a new test for you. 
This will help show how much Lnet bandwith you have from your single client.  There are tunable in the lnet later that can affect things.  Which QRD HCA are you using?

3. OBDFilter_survey :  Please see " 29.3. Testing OST Performance (obdfilter-survey)" in the Lustre manual.  This test will help demonstrate what the backed NVMe/ZFS setup can do at the OBD layer in Lustre.  

Thanks,
 Keith 
-----Original Message-----
From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Riccardo Veraldi
Sent: Thursday, August 17, 2017 10:48 PM
To: Dennis Nelson <dnelson at ddn.com>; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] Lustre poor performance

this is my lustre.conf

[drp-tst-ffb01:~]$ cat /etc/modprobe.d/lustre.conf options lnet networks=o2ib5(ib0),tcp5(enp1s0f0)

data transfer is over infiniband

ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 65520
        inet 172.21.52.83  netmask 255.255.252.0  broadcast 172.21.55.255


On 8/17/17 10:45 PM, Riccardo Veraldi wrote:
> On 8/17/17 9:22 PM, Dennis Nelson wrote:
>> It appears that you are running iozone on a single client?  What kind of network is tcp5?  Have you looked at the network to make sure it is not the bottleneck?
>>
> yes the data transfer is on ib0 interface and I did a memory to memory 
> test through InfiniBand QDR  resulting in 3.7GB/sec.
> tcp is used to connect to the MDS. It is tcp5 to differentiate it from 
> my other many Lustre clusters. I could have called it tcp but it does 
> not make any difference performance wise.
> I ran the test from one single node yes, I ran the same test also 
> locally on a zpool identical to the one on the Lustre OSS.
>  Ihave 4 identical servers each of them with the aame nvme disks:
>
> server1: OSS - OST1 Lustre/ZFS  raidz1
>
> server2: OSS - OST2 Lustre/ZFS  raidz1
>
> server3: local ZFS raidz1
>
> server4: Lustre client
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


More information about the lustre-discuss mailing list