[Lustre-discuss] Could "Client & MGS" or "Client & OSS" running at one node together?

[zhengfeng]

Dear all, thanks a lot for your answers ;)

Now I have another problem about the network between nodes.
Since there is no Infiniband or 10G-NIC, but I still want to 
increase the bandwidth by add more 1G-NICs, I plan to use Linux bonding.

Then, bonding 4 NICs together at one node, BUT there is NO performance 
enhanced no matter which bongding mode, described in kernel doc, used.
In stead, the performance of 4-NICs-bonding is lower than 1 NIC's.
Then we use 2 NICs bonding, the performance is better than 1 NIC's.
The result is :
bonding 2 NIC: 1 + 1 > 1
bonding 4 NIC: 1 + 1 + 1 + 1 < 1

So confused......
The benchmark we used is "netperf". 
And I use "tcpdump" to dump the packages, found that there are great 
of TCP segments out of orders.

My question is that:
a) TCP segments are out of order, which induced that 4-NIC-bonding
 performance decay, is this the root cause?
b) We are doubting the feasibility of this method: using 4-NIC-bonding 
to increase bandwidth. Any proposals about that?  If so, maybe I should 
use some other method instead of this.

Thanks again, all




Best Regards
Zheng 

From: Peter Grandi
Date: 2012-03-07 20:54
To: Lustre discussion
Subject: Re: [Lustre-discuss] Could "Client & MGS" or "Client & OSS" running at one node together?
> Since there is no more node in our project when using Lustre,
> I want to confirm that:

> 1) Could the "Client" and "MGS" run at one node together? or
> could "Client" and "OSS" run at one node together? 2) Suppose
> I had deployed them at one node, what potential shortcomings
> or harm are there?

Running MGS and MDS on the same nodes is customary, see:
   http://wiki.lustre.org/manual/LustreManual20_HTML/LustreOperations.html#50438194_24122

Running the MGS, MDS and OSS service on the same node is
possible and fairly common in very small setups, usually those
in which there is only 1-2 nodes.

It is possible to use the client code on all types of Lustre
servers, but at least in the case of using the client code on an
OSS there is the non-negligible possibility of a resource
deadlock, if the client uses the OSS on the same node, as the
client and OSS codes compete for memory, so in the past this has
been discouraged.

This is documented here:
  http://wiki.lustre.org/manual/LustreManual20_HTML/LustreOperations.html#50438194_84876
   «Caution - Do not do this when the client and OSS are on the
    same node, as memory pressure between the client and OSS can
    lead to deadlocks.»
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20120308/e58089b6/attachment.htm>