[Lustre-discuss] [HPDD-discuss] Same performance Infiniband and Ethernet

[Dilger, Andreas]

Are you sure your client is using the IB network?  What does:

    lctl get_param osc.*.imports

report for the NIDs of the OSTs? If it is still using the TCP connection you may need to run writeconf to fix the client config log. 

Cheers, Andreas

> On May 19, 2014, at 4:34, "Pardo Diaz, Alfonso" <alfonso.pardo at ciemat.es> wrote:
> 
> Hi,
> 
> I have migrated my Lustre 2.2 to 2.5.1 and I have equipped my OSS/MDS and clients with Infiniband QDR interfaces.
> I have compile lustre with OFED 3.2 and I have configured lnet module with:
> 
> options lent networks=“o2ib(ib0),tcp(eth0)”
> 
> 
> But when I try to compare the lustre performance across Infiniband (o2ib), I get the same performance than across ethernet (tcp):
> 
> INFINIBAND TEST:
> dd if=/dev/zero of=test.dat bs=1M count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes (1,0 GB) copied, 5,88433 s, 178 MB/s
> 
> ETHERNET TEST:
> dd if=/dev/zero of=test.dat bs=1M count=1000
> 1000+0 records in
> 1000+0 records out
> 1048576000 bytes (1,0 GB) copied, 5,97423 s, 154 MB/s
> 
> 
> And this is my scenario:
> 
> - 1 MDs with SSD RAID10 MDT
> - 10 OSS with 2 OST per OSS
> - Infiniband interface in connected mode
> - Centos 6.5
> - Lustre 2.5.1
> - Striped filesystem “lfs setstripe -s 1M -c 10"
> 
> 
> I know my infiniband running correctly, because if I use IPERF3 between client and servers I got 40Gb/s by infiniband and 1Gb/s by ethernet connections.
> 
> 
> 
> Could you help me?
> 
> 
> Regards,
> 
> 
> 
> 
> 
> Alfonso Pardo Diaz
> System Administrator / Researcher
> c/ Sola nº 1; 10200 Trujillo, ESPAÑA
> Tel: +34 927 65 93 17 Fax: +34 927 32 32 37
> 
> 
> 
> 
> ----------------------------
> Confidencialidad: 
> Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario y puede contener información privilegiada o confidencial. Si no es vd. el destinatario indicado, queda notificado de que la utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente respondiendo al mensaje y proceda a su destrucción.
> 
> Disclaimer: 
> This message and its attached files is intended exclusively for its recipients and may contain confidential information. If you received this e-mail in error you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited and may be unlawful. In this case, please notify us by a reply and delete this email and its contents immediately. 
> ----------------------------
> 
> _______________________________________________
> HPDD-discuss mailing list
> HPDD-discuss at lists.01.org
> https://lists.01.org/mailman/listinfo/hpdd-discuss