[lustre-discuss] Lustre poor performance

Riccardo Veraldi Riccardo.Veraldi at cnaf.infn.it
Thu Aug 17 22:48:19 PDT 2017


this is my lustre.conf

[drp-tst-ffb01:~]$ cat /etc/modprobe.d/lustre.conf
options lnet networks=o2ib5(ib0),tcp5(enp1s0f0)

data transfer is over infiniband

ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 65520
        inet 172.21.52.83  netmask 255.255.252.0  broadcast 172.21.55.255


On 8/17/17 10:45 PM, Riccardo Veraldi wrote:
> On 8/17/17 9:22 PM, Dennis Nelson wrote:
>> It appears that you are running iozone on a single client?  What kind of network is tcp5?  Have you looked at the network to make sure it is not the bottleneck?
>>
> yes the data transfer is on ib0 interface and I did a memory to memory
> test through InfiniBand QDR  resulting in 3.7GB/sec.
> tcp is used to connect to the MDS. It is tcp5 to differentiate it from
> my other many Lustre clusters. I could have called it tcp but
> it does not make any difference performance wise.
> I ran the test from one single node yes, I ran the same test also
> locally on a zpool identical to the one on the Lustre OSS.
>  Ihave 4 identical servers each of them with the aame nvme disks:
>
> server1: OSS - OST1 Lustre/ZFS  raidz1
>
> server2: OSS - OST2 Lustre/ZFS  raidz1
>
> server3: local ZFS raidz1
>
> server4: Lustre client
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org




More information about the lustre-discuss mailing list