[Lustre-discuss] Lustre with 10GbE or Infiniband?

[Brian J. Murrell]

On Wed, 2009-02-11 at 11:08 -0800, Jeffrey Bennett wrote:
> Hi,
> 
> Has anybody done any performance comparison between Lustre with 10GbE and Lustre with Infiniband 4X SDR? I wonder if they perform similarly.

While I don't have any performance numbers or experience for you, I will
mention the differences in the way Lustre uses those two technologies.

On 10GbE, Lustre (via it's sock LND) will use the TCP/IP stack on top of
the ethernet stack.  With Infiniband, we communicate directly with the
I/B stack (via the o2ib LND) and take direct advantage of it's RDMA
capabilities to achieve a very high percentage of wire speed.

My gut feeling is that the overhead of TCP/IP carves some percentage out
of your ability to achieve full wire speed.

Maybe some others here, including our benchmarking folks here at Sun can
provide some real world experiences and comparisons.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090211/d1810e5a/attachment.pgp>