<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple style='word-wrap: break-word;
-webkit-nbsp-mode: space;-webkit-line-break: after-white-space'>
<div class=Section1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>If the intention is not size, but to spread your I/Os on as many
spindles as possible, you could still have these volume groups. Once you
create these volumes, you could have them sliced into multiple LUNs where their
collective sizes are acceptable by EXT3. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Regards<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>-Peter<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
lustre-discuss-bounces@clusterfs.com
[mailto:lustre-discuss-bounces@clusterfs.com] <b>On Behalf Of </b>Aaron Knister<br>
<b>Sent:</b> Wednesday, October 17, 2007 7:30 PM<br>
<b>To:</b> Andreas Dilger<br>
<b>Cc:</b> lustre-discuss@clusterfs.com; Lundgren, Andrew<br>
<b>Subject:</b> Re: [Lustre-discuss] Hardware Question<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<p class=MsoNormal>So if I have arrays with 15 drives in them should I just
configure two smaller arrays? Also if I make a giant 30 terabyte filesystem of
underlying say 6TB disk arrays and one of my disk arrays bites the dust what
happens to the rest of the filesystem and how easy is it to recover from this
situation?<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>-Aaron<o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<div>
<p class=MsoNormal>On Oct 10, 2007, at 11:48 AM, Andreas Dilger wrote:<o:p></o:p></p>
</div>
<p class=MsoNormal><br>
<br>
<o:p></o:p></p>
<div>
<p class=MsoNormal>On Oct 10, 2007<span class=apple-converted-space> </span>09:40
-0600, Lundgren, Andrew wrote:<o:p></o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>As RH 5.1 will support 16TB ext3 partitions, will lustre
inherit that<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>functionality?<span class=apple-converted-space> </span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>We haven't looked at this yet.<span
class=apple-converted-space> </span>The ldiskfs code is ext3 + patches,
so there<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>is some chance that it will work (more likely on 64-bit
platforms), but<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>we haven't audited the ldiskfs patches to check if they are
32-bit clean.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>-----Original Message-----<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>From: <a href="mailto:lustre-discuss-bounces@clusterfs.com">lustre-discuss-bounces@clusterfs.com</a><span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>[<a href="mailto:lustre-discuss-bounces@clusterfs.com">mailto:lustre-discuss-bounces@clusterfs.com</a>]
On Behalf Of<span class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Andreas Dilger<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Sent: Wednesday, October 10, 2007 9:26 AM<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>To: Aaron Knister<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Cc: <a href="mailto:lustre-discuss@clusterfs.com">lustre-discuss@clusterfs.com</a><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Subject: Re: [Lustre-discuss] Hardware Question<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>On Oct 06, 2007<span class=apple-converted-space> </span>10:28
-0400, Aaron Knister wrote:<o:p></o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>Oh, right I forgot about that. Well...if i had an 8tb lun<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class=MsoNormal>and split it<span class=apple-converted-space> </span><o:p></o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>into 2 volume groups using LVM do you think the performance<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
</blockquote>
<div>
<p class=MsoNormal>would be<span class=apple-converted-space> </span><o:p></o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>worse than making 2 raids at the hardware level?<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Well, it won't be doing the disks any favours, since you'll<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>now have contention between the OSTs, and the kernel will be<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>doing a poor job with the IO elevator decisions.<span
class=apple-converted-space> </span>I would<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>suggest making 2 smaller RAID LUNs instead.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>In the end it is up to you to decide if the IO performance
is<span class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>acceptable.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>You can do some testing using lustre-iokit to see what the<span
class=apple-converted-space> </span><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>component device performance is.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>On Oct 5, 2007, at 6:18 PM, Andreas Dilger wrote:<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>On Oct 05, 2007<span class=apple-converted-space> </span>13:14
-0400, Aaron Knister wrote:<o:p></o:p></p>
</div>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<div>
<p class=MsoNormal>Make that 6x 9.7TB luns.<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Lustre (== ext3) doesn't support >= 8TB LUNs.<o:p></o:p></p>
</div>
</blockquote>
</blockquote>
</blockquote>
</blockquote>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Cheers, Andreas<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>--<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Andreas Dilger<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Principal Software Engineer<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal>Cluster File Systems, Inc.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'>Aaron Knister<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'>Associate Systems Administrator/Web Designer<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'>Center for Research on Environment and Water<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'><o:p> </o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'>(301) 595-7001<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'><a href="mailto:aaron@iges.org">aaron@iges.org</a><o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'><o:p> </o:p></span></p>
</div>
<p class=MsoNormal><span style='font-size:7.0pt;font-family:"Helvetica","sans-serif";
color:black'><br>
<br>
</span><o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>