[Lustre-discuss] Multiple IB ports

[Andreas Dilger]

On 2011-03-21, at 4:53 AM, Brian O'Connor wrote:
>     Any body actually using multiple IB ports on a client for an aggregated connection?
>  
> Ie. Many oss with one qdr IB each. Clients with 4 qdr IB ports. Assuming the
> normal issues with bus bandwidth etc, what sort of perf can I expect
>  
> qdr ~ 3-4Gbytes/Sec
>  
> I’m trying to size a cluster and clients to get ~10GBytes/Sec on *one*
> client node.

I believe this is possible to some limited extent today.  The main issue is that the primary NID addresses for the OST IB cards need to be on different subnets so that the clients will route the traffic to the OSTs via the different IB HCAs.

I don't have low-level details on this myself, but I believe there are a couple of sites that have done this.

> If I can aggregate IB linearly the next step will be to try and figure out
> How to get 10Gigabytes/s to local storage L
>  
>  
> Some times customers are crazy…….
>  
>  
>  
> Brian O'Connor
>  
> -------------------------------------------------
>  
> SGI Consulting
>  
> Email: briano at sgi.com, Mobile +61 417 746 452
>  
> Phone: +61 3 9963 1900, Fax: +61 3 9963 1902
>  
> 357 Camberwell Road, Camberwell, Victoria, 3124
>  
> AUSTRALIA http://www.sgi.com/support/services
>  
> -------------------------------------------------
>  
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss


Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.