<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=utf-8">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Times New Roman","serif";
color:blue;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-GB link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='color:blue'>The file size is computed from the
sizes of all its stripes to avoid<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'>unnecessary communication with the
MDS while a file is being<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'>extended.<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='color:blue'>Obviously the next question is “does
that mean you have to consult all<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'>the OSTs whenever you stat a file?”
and currently the answer is yes.<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'>However we are developing a new
feature to allow Lustre to cache the<o:p></o:p></span></p>
<p class=MsoNormal><span style='color:blue'>sizes of inactive files on the MDS
to give us the best of both worlds.<o:p></o:p></span></p>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span
style='color:blue'>Cheers, <br>
Eric <o:p></o:p></span></p>
</blockquote>
<p class=MsoNormal><span style='color:blue'><o:p> </o:p></span></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:
"Tahoma","sans-serif"'>From:</span></b><span lang=EN-US style='font-size:10.0pt;
font-family:"Tahoma","sans-serif"'> lustre-devel-bounces@lists.lustre.org
[mailto:lustre-devel-bounces@lists.lustre.org] <b>On Behalf Of </b>office@hailoo.com<br>
<b>Sent:</b> 11 August 2009 12:32 AM<br>
<b>To:</b> lustre-devel@lists.lustre.org<br>
<b>Subject:</b> [Lustre-devel] Role of the Metadata Server during File I/O<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>One of Lustre's key innovations seems to be the separation
of metadata from file data. According to Sun, "Lustre file
operations bypass the MetaData server completely and fully utilize the parallel
data paths to all OSSs in the cluster." (See http://www.sun.com/software/products/lustre/features.xml)
<o:p></o:p></p>
<p>However, can this really be the case? <o:p></o:p></p>
<p>In POSIX-compliant file I/O, a call to write() that starts at an offset
which is greater than the file size must write zeroes to disk to make up for
the missing space. So if you have a file size of 0 bytes, and then you
write a single byte at offset 10, bytes 0 through 9 of the file will contain
zeros. <o:p></o:p></p>
<p>But if a file on a Lustre system is striped across multiple OSTs, how does
Lustre avoid communicating with the Metadata Server at every write
operation? Consider the following scenario: <o:p></o:p></p>
<p>You have 4 OSTs and you create a new file and stripe it across all 4 OSTs,
and you set the stripe size to 4 bytes. (I know that is too small but I'm
just keeping this simple.) <o:p></o:p></p>
<p>Now, suppose you call write() and write 1 byte to the file at offset
5. Lustre must now write 4 zero bytes on the first OST, and 1 non-zero
byte on the second OST. But in order to know that it is necessary to
write zeroes to the first OST, the client would need access to global
information about the total size of the file. Therefore, wouldn't it need
to check with the Metadata Server to determine the total file size before every
call to write()? <o:p></o:p></p>
<p>Any information anyone can provide me on the implementation details/strategies
used here would be greatly appreciated. <o:p></o:p></p>
<p>-Charles Salvia <o:p></o:p></p>
</div>
</div>
</body>
</html>