<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=utf-8">
<STYLE>
BLOCKQUOTE {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px; MARGIN-LEFT: 2em
}
OL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
UL {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
BODY {
FONT-SIZE: 10.5pt; COLOR: #000080; LINE-HEIGHT: 1.5; FONT-FAMILY: 宋体
}
</STYLE>
<META content="MSHTML 6.00.2900.6182" name=GENERATOR></HEAD>
<BODY style="MARGIN: 10px">
<DIV>Dear all, thanks a lot for your answers ;)</DIV>
<DIV> </DIV>
<DIV>Now I have another problem about the network between nodes.</DIV>
<DIV>Since there is no Infiniband or 10G-NIC, but I still want to
</DIV>
<DIV>increase the bandwidth by add more 1G-NICs, I plan to use Linux
bonding.</DIV>
<DIV> </DIV>
<DIV>Then, bonding 4 NICs together at one node, BUT there is <U
style="FONT-WEIGHT: bold">NO</U> performance </DIV>
<DIV>enhanced no matter which bongding mode, described in kernel doc,
used.</DIV>
<DIV>In stead, the performance of 4-NICs-bonding is lower than 1 NIC's.</DIV>
<DIV>Then we use 2 NICs bonding, the performance is better than 1 NIC's.</DIV>
<DIV>The result is :</DIV>
<DIV style="TEXT-INDENT: 2em">bonding 2 NIC: 1 + 1 <SPAN
style="FONT-WEIGHT: bold">></SPAN> 1</DIV>
<DIV style="TEXT-INDENT: 2em">bonding 4 NIC: 1 + 1 + 1 + 1 <SPAN
style="FONT-WEIGHT: bold"><</SPAN> 1</DIV>
<DIV> </DIV>
<DIV>So confused......</DIV>
<DIV>The benchmark we used is "netperf". </DIV>
<DIV>And I use "tcpdump" to dump the packages, found that there are great
</DIV>
<DIV>of TCP segments out of orders.</DIV>
<DIV> </DIV>
<DIV>My question is that:</DIV>
<DIV style="TEXT-INDENT: 2em">a) TCP segments are out of order,
which induced that 4-NIC-bonding</DIV>
<DIV style="TEXT-INDENT: 2em"> performance decay, is this the root
cause?</DIV>
<DIV style="TEXT-INDENT: 2em">b) We are doubting the feasibility of this method:
using 4-NIC-bonding </DIV>
<DIV style="TEXT-INDENT: 2em">to increase bandwidth. Any proposals about
that? If so, maybe I should </DIV>
<DIV style="TEXT-INDENT: 2em">use some other method instead of this.</DIV>
<DIV style="TEXT-INDENT: 2em"> </DIV>
<DIV>Thanks again, all</DIV>
<DIV style="TEXT-INDENT: 2em"> </DIV>
<DIV>
<HR style="WIDTH: 210px; HEIGHT: 1px" align=left color=#b5c4df SIZE=1>
</DIV>
<DIV>
<DIV style="FONT-WEIGHT: bold; FONT-STYLE: italic">Best Regards</DIV></DIV>
<DIV><SPAN style="FONT-WEIGHT: bold; FONT-STYLE: italic">Zheng </SPAN></DIV>
<DIV><SPAN style="FONT-WEIGHT: bold; FONT-STYLE: italic"></SPAN> </DIV>
<DIV
style="BORDER-RIGHT: medium none; PADDING-RIGHT: 0cm; BORDER-TOP: #b5c4df 1pt solid; PADDING-LEFT: 0cm; PADDING-BOTTOM: 0cm; BORDER-LEFT: medium none; PADDING-TOP: 3pt; BORDER-BOTTOM: medium none">
<DIV
style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; COLOR: #000000; PADDING-TOP: 8px">
<DIV><B>From:</B> <A href="mailto:pg_lus@lus.for.sabi.co.uk ">Peter
Grandi</A></DIV>
<DIV><B>Date:</B> 2012-03-07 20:54</DIV>
<DIV><B>To:</B> <A href="mailto:lustre-discuss@lists.Lustre.org">Lustre
discussion</A></DIV>
<DIV><B>Subject:</B> Re: [Lustre-discuss] Could "Client & MGS" or
"Client & OSS" running at one node together?</DIV></DIV></DIV>
<DIV>
<DIV>> Since there is no more node in our project when using Lustre,</DIV>
<DIV>> I want to confirm that:</DIV>
<DIV> </DIV>
<DIV>> 1) Could the "Client" and "MGS" run at one node together? or</DIV>
<DIV>> could "Client" and "OSS" run at one node together? 2) Suppose</DIV>
<DIV>> I had deployed them at one node, what potential shortcomings</DIV>
<DIV>> or harm are there?</DIV>
<DIV> </DIV>
<DIV>Running MGS and MDS on the same nodes is customary, see:</DIV>
<DIV> http://wiki.lustre.org/manual/LustreManual20_HTML/LustreOperations.html#50438194_24122</DIV>
<DIV> </DIV>
<DIV>Running the MGS, MDS and OSS service on the same node is</DIV>
<DIV>possible and fairly common in very small setups, usually those</DIV>
<DIV>in which there is only 1-2 nodes.</DIV>
<DIV> </DIV>
<DIV>It is possible to use the client code on all types of Lustre</DIV>
<DIV>servers, but at least in the case of using the client code on an</DIV>
<DIV>OSS there is the non-negligible possibility of a resource</DIV>
<DIV>deadlock, if the client uses the OSS on the same node, as the</DIV>
<DIV>client and OSS codes compete for memory, so in the past this has</DIV>
<DIV>been discouraged.</DIV>
<DIV> </DIV>
<DIV>This is documented here:</DIV>
<DIV> http://wiki.lustre.org/manual/LustreManual20_HTML/LustreOperations.html#50438194_84876</DIV>
<DIV> «Caution - Do not do this when the client and OSS are on the</DIV>
<DIV> same node, as memory pressure between the client and OSS can</DIV>
<DIV> lead to deadlocks.»</DIV>
<DIV>_______________________________________________</DIV>
<DIV>Lustre-discuss mailing list</DIV>
<DIV>Lustre-discuss@lists.lustre.org</DIV>
<DIV>http://lists.lustre.org/mailman/listinfo/lustre-discuss</DIV></DIV></BODY></HTML>