[lustre-discuss] Lustre over 10 Gb Ethernet with and without RDMA

Ben Evans bevans at cray.com
Fri Jun 19 06:46:44 PDT 2015


It is faster, but I don’t know what price/performance tradeoff is, as I only used it as an engineer.

As an alternative, take a look at RoCE, it does much the same thing but uses normal (?) hardware.  It’s still pretty new, though, so you might have some speedbumps.

-Ben Evans

From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of INKozin
Sent: Friday, June 19, 2015 5:43 AM
To: lustre-discuss at lists.lustre.org
Subject: [lustre-discuss] Lustre over 10 Gb Ethernet with and without RDMA

My question is about performance advantages of Lustre RDMA over 10 Gb Ethernet. When using 10 Gb Ethernet to build Lustre, is it worth paying the premium for iWARP? I understand that iWARP essentially reduces latency but less sure of its specific implications for storage. Would it improve performance on small files? Any pointers to representative benchmarks will be very appreciated.

Celsio has released a white paper in which they compare Lustre RDMA over 40 Gb Ethernet and FDR IB
http://www.chelsio.com/wp-content/uploads/resources/Lustre-Over-iWARP-vs-IB-FDR.pdf
where they claim comparable performance of both.
How much worse the throughput on small block sizes would be without iWARP?

Thank you
Igor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20150619/2b73a546/attachment.htm>


More information about the lustre-discuss mailing list