<HTML>
<HEAD>
<TITLE>Re: [Lustre-discuss] Advice on system configuration</TITLE>
</HEAD>
<BODY>
<FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'><BR>
Hi Iain,<BR>
<BR>
One of the clusters I work with here uses jumbo-enabled LACP-aggregated NICs, and I can get about 350 MB/s total sustained reads out of two OSSes (each OSS has two NICs, so 4 GigE paths total).<BR>
<BR>
However, the cluster itself is smaller than yours; you might want to investigate another network fabric, as GigE performance may start to suffer with that many nodes due to the limitations of Ethernet itself.<BR>
<BR>
A good L3 switch (Extreme, Cisco, Foundry, etc.) will be able to provide you with optimal performance.<BR>
<BR>
hth,<BR>
Klaus<BR>
<BR>
On 2/12/08 1:45 AM, "Iain Grant" <Iain.Grant@scri.ac.uk>did etch on stone tablets:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT SIZE="4"><FONT FACE="Arial"><SPAN STYLE='font-size:13.0px'>We have an in house 27 node cluster. With no shared storage capability.<BR>
<BR>
I have just order 2 storage nodes with extra drives in them and was hoping I could use this with Lustre and provide a good shared storage setup.<BR>
<BR>
How would you suggest configuring this setup ?<BR>
<BR>
Each node comes with dual network adapters so I was thinking about bonding them together to provide more bandwidth ( our switch should allow us to do this.)<BR>
<BR>
Thanks<BR>
<BR>
Iain<BR>
</SPAN></FONT><SPAN STYLE='font-size:13.0px'><FONT COLOR="#800000"><FONT FACE="Times New Roman"><B>Iain Grant</B></FONT></FONT></SPAN></FONT><FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'> <BR>
</SPAN><FONT COLOR="#008000"><FONT SIZE="4"><SPAN STYLE='font-size:13.0px'><B><I>Linux Administrator</I></B></SPAN></FONT></FONT><SPAN STYLE='font-size:12.0px'> <BR>
</SPAN><FONT SIZE="1"><SPAN STYLE='font-size:9.0px'>Scottish Crop Research Institute <BR>
Dundee DD2 5DA <BR>
Tel: +44 (0)1382 562731 <BR>
<a href="mailto:Iain.Grant@scri.ac.uk">mailto:Iain.Grant@scri.ac.uk</a></SPAN></FONT><SPAN STYLE='font-size:12.0px'> <a href="mailto:Iain.Grant@scri.ac.uk"><mailto:Iain.Grant@scri.ac.uk></a> </SPAN><FONT SIZE="1"><SPAN STYLE='font-size:9.0px'> <BR>
</SPAN></FONT><SPAN STYLE='font-size:12.0px'><BR>
</SPAN></FONT><FONT SIZE="1"><FONT FACE="Times New Roman"><SPAN STYLE='font-size:9.0px'>www.scri.ac.uk</SPAN></FONT></FONT><FONT FACE="Times New Roman"><FONT SIZE="5"><SPAN STYLE='font-size:16.0px'> <a href="http://www.scri.ac.uk/"><http://www.scri.ac.uk/></a> <BR>
</SPAN></FONT></FONT><FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'><BR>
</SPAN></FONT><FONT SIZE="5"><FONT FACE="Times New Roman"><SPAN STYLE='font-size:16.0px'> <BR>
</SPAN></FONT></FONT><FONT FACE="Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12.0px'><BR>
<BR>
<BR>
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _<BR>
<BR>
SCRI, Invergowrie, Dundee, DD2 5DA. <BR>
The Scottish Crop Research Institute is a charitable company limited by guarantee. <BR>
Registered in Scotland No: SC 29367.<BR>
Recognised by the Inland Revenue as a Scottish Charity No: SC 006662.<BR>
<BR>
<BR>
DISCLAIMER:<BR>
<BR>
This email is from the Scottish Crop Research Institute, but the views <BR>
expressed by the sender are not necessarily the views of SCRI and its <BR>
subsidiaries. This email and any files transmitted with it are confidential <BR>
to the intended recipient at the e-mail address to which it has been <BR>
addressed. It may not be disclosed or used by any other than that addressee.<BR>
If you are not the intended recipient you are requested to preserve this <BR>
confidentiality and you must not use, disclose, copy, print or rely on this <BR>
e-mail in any way. Please notify postmaster@scri.ac.uk quoting the <BR>
name of the sender and delete the email from your system.<BR>
<BR>
Although SCRI has taken reasonable precautions to ensure no viruses are <BR>
present in this email, neither the Institute nor the sender accepts any <BR>
responsibility for any viruses, and it is your responsibility to scan the email <BR>
and the attachments (if any).<BR>
<BR>
<HR ALIGN=CENTER SIZE="3" WIDTH="95%"></SPAN></FONT><FONT SIZE="2"><FONT FACE="Monaco, Courier New"><SPAN STYLE='font-size:10.0px'>_______________________________________________<BR>
Lustre-discuss mailing list<BR>
Lustre-discuss@lists.lustre.org<BR>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><BR>
</SPAN></FONT></FONT></BLOCKQUOTE><FONT SIZE="2"><FONT FACE="Monaco, Courier New"><SPAN STYLE='font-size:10.0px'><BR>
</SPAN></FONT></FONT>
</BODY>
</HTML>