<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:o =
"urn:schemas-microsoft-com:office:office" xmlns:w =
"urn:schemas-microsoft-com:office:word" xmlns:st1 =
"urn:schemas-microsoft-com:office:smarttags"><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.6000.17092" name=GENERATOR><o:SmartTagType
name="country-region"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="Street"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="State"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="address"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="City"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><o:SmartTagType
name="place"
namespaceuri="urn:schemas-microsoft-com:office:smarttags"></o:SmartTagType><!--[if !mso]>
<STYLE>
st1\:*{behavior:url(#default#ieooui) }
</STYLE>
<![endif]-->
<STYLE>
<!--
/* Font Definitions */
@font-face
{font-family:Courier;
panose-1:2 7 4 9 2 2 5 2 4 4;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:Arial;
color:windowtext;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.Section1
{page:Section1;}
-->
</STYLE>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff><SPAN
class=780281009-21032011>Hi Brian,</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff><SPAN
class=780281009-21032011></SPAN></FONT> </DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff><SPAN
class=780281009-21032011>From my understanding, but confirmation from more
skilled people on the list would be welcomed, using multiple IB
ports with a lustre client will be difficult to manage, and will probably
not bring any performance improvements.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff><SPAN
class=780281009-21032011>I was told by a colleague that there were currently too
many internal locks in the clients to sustain a big throughput. Lustre is
designed for global throughput on many clients, but not on individual
clients.</SPAN></FONT></DIV>
<DIV dir=ltr align=left><FONT face=Arial color=#0000ff><SPAN
class=780281009-21032011>I can observe this on my site, where I have enough
storage and servers to reach 21GB/s globally, but am unable to get more than
300MB/s on a single client even though the DDR IB network would sustain +800MB/s
...</SPAN></FONT></DIV><BR>
<BLOCKQUOTE dir=ltr
style="PADDING-LEFT: 5px; MARGIN-LEFT: 5px; BORDER-LEFT: #0000ff 2px solid; MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> lustre-discuss-bounces@lists.lustre.org
[mailto:lustre-discuss-bounces@lists.lustre.org] <B>On Behalf Of </B>Brian
O'Connor<BR><B>Sent:</B> lundi 21 mars 2011 04:53<BR><B>To:</B>
lustre-discuss@lists.lustre.org<BR><B>Subject:</B> [Lustre-discuss] Multiple
IB ports<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hi,<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"> Any body
actually using multiple IB ports on a client for an aggregated
connection?<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Ie. Many oss with one qdr IB each.
Clients with 4 qdr IB ports. Assuming the normal<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">issues with bus bandwidth etc,
what sort of perf can I expect <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">qdr ~
3-4Gbytes/Sec<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">I’m trying to size a cluster and
clients to get ~10GBytes/Sec on *<B><SPAN
style="FONT-WEIGHT: bold">one</SPAN></B>* <o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">client node.
<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">If I can aggregate IB linearly the
next step will be to try and figure out<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">How to get 10Gigabytes/s to local
storage </SPAN></FONT><FONT face=Wingdings size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Wingdings">L</SPAN></FONT><FONT
face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Some times customers are
crazy…….<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">Brian
O'Connor</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">-------------------------------------------------</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">SGI
Consulting</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">Email: </SPAN></FONT><A
href="mailto:briano@sgi.com"><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">briano@sgi.com</SPAN></FONT></A><FONT
face=Courier size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">,
<st1:City w:st="on"><st1:place w:st="on">Mobile</st1:place></st1:City> +61 417
746 452</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">Phone: +61 3 9963 1900, Fax: +61
3 9963 1902</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><st1:address w:st="on"><st1:Street w:st="on"><FONT
face=Courier size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">357
Camberwell Road</SPAN></FONT></st1:Street><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">, <st1:City
w:st="on">Camberwell</st1:City>, <st1:State
w:st="on">Victoria</st1:State></SPAN></FONT></st1:address><FONT face=Courier
size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">, 3124
</SPAN></FONT><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><st1:country-region w:st="on"><st1:place w:st="on"><FONT
face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">AUSTRALIA</SPAN></FONT></st1:place></st1:country-region><FONT
face=Courier size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">
</SPAN></FONT><A href="http://www.sgi.com/support/services"><FONT face=Courier
size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">http://www.sgi.com/support/services</SPAN></FONT></A><o:p></o:p></P>
<DIV>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"> <o:p></o:p></SPAN></FONT></P></DIV>
<P class=MsoNormal><FONT face=Courier size=2><SPAN
style="FONT-SIZE: 10pt; FONT-FAMILY: Courier">-------------------------------------------------</SPAN></FONT><o:p></o:p></P>
<P class=MsoNormal><FONT face="Times New Roman" size=3><SPAN
style="FONT-SIZE: 12pt"><o:p> </o:p></SPAN></FONT></P></DIV></BLOCKQUOTE></BODY></HTML>