[Lustre-discuss] tcp network load balancing understanding lustre 1.8

[Arden Wiebe]

Michael,

This might help answer some questions.  http://ioio.ca/Lustre-tcp-bonding/OST2.png which shows my mostly not tuned OSS and OST's pulling 400+MiB/s over TCP Bonding provided by the kernel complete with a cat of the modeprobe.conf file.  You have the other links I've sent you but the picture above is relevant to your questions. 

Arden

--- On Thu, 5/7/09, Michael Ruepp <michael at schwarzfilm.ch> wrote:

> From: Michael Ruepp <michael at schwarzfilm.ch>
> Subject: [Lustre-discuss] tcp network load balancing understanding lustre 1.8
> To: lustre-discuss at lists.lustre.org
> Date: Thursday, May 7, 2009, 5:50 AM
> Hi there,
> 
> I am configured a simple tcp lustre 1.8 with one mdc (one
> nic) and two  
> oss (four nic per oss)
> As well as in the 1.6 documentation, the multihomed
> sections is a  
> little bit unclear to me.
> 
> I give every NID a IP in the same subnet, eg:
> 10.111.20.35-38 - oss0  
> and 10.111.20.39-42 oss1
> 
> Do I have to make modprobe.conf.local look like this to
> force lustre  
> to use all four interfaces parallel:
> 
> options lnet networks=tcp0(eth0,eth1,eth2,eth3)
> Because on Page 138 the 1.8 Manual says:
> "Note – In the case of TCP-only clients, the first
> available non- 
> loopback IP interface
> is used for tcp0 since the interfaces are not specified. "
> 
> or do I have to specify it like this:
> options lnet networks=tcp
> Because on Page 112 the lustre 1.6 Manual says:
> "Note – In the case of TCP-only clients, all available IP
> interfaces  
> are used for tcp0
> since the interfaces are not specified. If there is more
> than one, the  
> IP of the first one
> found is used to construct the tcp0 ID."
> 
> Which is the opposite of the 1.8 Manual
> 
> My goal ist to let lustre utilize all four Gb Links
> parallel. And my  
> Lustre Clients are equipped with two Gb links which should
> be utilized  
> by the lustre clients as well (eth0, eth1)
> 
> Or is bonding the better solution in terms of performance?
> 
> Thanks very much for input,
> 
> Michael Ruepp
> Schwarzfilm AG
> 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>