[Lustre-discuss] Bonded client interfaces and 10GbE server

[Klaus Steden]

Hi Ralf,

The specification for 802.3ad does not permit the striping of a single data
path across multiple links, i.e. a single TCP/UDP conversation takes place
with a lone physical interface, the TCP/IP stack does not split it apart so
it can use multiple paths.

If you use a single 10GigE link instead multiple GigE (assuming you have a
fast performing NIC that supports RDMA), you would see Gbit+ throughput for
a single conversation. However, both peers would need to be using 10GigE
NICs.

LACP bonding only provides more aggregate bandwidth over a given link, it
does not double (triple, quadruple, etc.) to a single thread of
communication without some application-specific or hardware-specific
optimizations.

hth,
Klaus

On 7/7/09 6:44 AM, "Ralf Utermann" <ralf.utermann at physik.uni-augsburg.de>
etched on stone tablets:

> Dear list,
> 
> we have setup of OSS and some clients with a dual Gigabit
> trunk (miimon=100 mode=802.3ad xmit_hash_policy=layer3+4).
> If the clients stripe over targets on different OSS, they see
> a dual link bandwidth. If however, they stripe over targets on
> the same OSS, they only get the bandwith of one link.
> 
> If I would attach the OSS with a single 10GbE link, could
> a client then use the second link, when striping over targets
> on same OSS?
> 
> Regards, Ralf