[lustre-discuss] Lustre poor performance
Riccardo Veraldi
Riccardo.Veraldi at cnaf.infn.it
Thu Aug 17 22:48:19 PDT 2017
this is my lustre.conf
[drp-tst-ffb01:~]$ cat /etc/modprobe.d/lustre.conf
options lnet networks=o2ib5(ib0),tcp5(enp1s0f0)
data transfer is over infiniband
ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 65520
inet 172.21.52.83 netmask 255.255.252.0 broadcast 172.21.55.255
On 8/17/17 10:45 PM, Riccardo Veraldi wrote:
> On 8/17/17 9:22 PM, Dennis Nelson wrote:
>> It appears that you are running iozone on a single client? What kind of network is tcp5? Have you looked at the network to make sure it is not the bottleneck?
>>
> yes the data transfer is on ib0 interface and I did a memory to memory
> test through InfiniBand QDR resulting in 3.7GB/sec.
> tcp is used to connect to the MDS. It is tcp5 to differentiate it from
> my other many Lustre clusters. I could have called it tcp but
> it does not make any difference performance wise.
> I ran the test from one single node yes, I ran the same test also
> locally on a zpool identical to the one on the Lustre OSS.
> Ihave 4 identical servers each of them with the aame nvme disks:
>
> server1: OSS - OST1 Lustre/ZFS raidz1
>
> server2: OSS - OST2 Lustre/ZFS raidz1
>
> server3: local ZFS raidz1
>
> server4: Lustre client
>
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
More information about the lustre-discuss
mailing list