<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
On 10/05/2011 09:06 AM, Nathan Rutman wrote:
<blockquote
cite="mid:73AED5C780AE05478241DB067651A92102378850@XYUS-EX22.xyus.xyratex.com"
type="cite"><br>
<div>
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000"> <br>
<blockquote
cite="mid:73AED5C780AE05478241DB067651A9210237883C@XYUS-EX22.xyus.xyratex.com"
type="cite">
<div>
<div style="background-color: transparent;"><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">Common<br>
Object ID</span><br>
<div style="font-family: Times; margin-left: 36pt;
margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">The
MDT<br>
tracks a special range of OST object ID’s (“wide
stripe<br>
objectid” = WSO) that are used on all OSTs. The
MDT<br>
assigns the next available WSO to the file, and
this<br>
objectid is used on all the OSTs. The OSTs must
never use<br>
these objects for regular striped files. A
special<br>
precreation group for these objects is probably
necessary,<br>
as well as orphan cleanup (the MDT should purge
"hole"<br>
objects that aren’t allocated from a particular
OST). The<br>
MDT should track the last assigned WSO; this will
be the<br>
starting point for new wide striped files after
recovery.</span><span class="Apple-style-span"
style="font-family: Helvetica;"><span
style="font-size: 11pt; font-family: Arial;
color: rgb(0, 0, 0); background-color:
transparent; font-weight: normal; font-style:
normal; font-variant: normal; text-decoration:
none; vertical-align: baseline; white-space:
pre-wrap;"> Objects cannot be migrated<br>
from one OST to another, since this would result
in<br>
out-of-order access. Similarly, s</span></span><span
class="Apple-style-span" style="font-family:
Helvetica;"><span class="Apple-style-span"
style="font-family: Arial; font-size: 15px;
white-space: pre-wrap;">tripes can<br>
never be added to holes.</span></span></div>
<span style="font-size: 11pt; font-family: Arial;
color: rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">FID-on-OST</span><br>
<div style="font-family: Times; margin-left: 36pt;
margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">Use
a<br>
mapping of the MDT FID to uniquely determine an
OST<br>
object. The clients and MDT add in the OST number
to the<br>
MDT FID (probably just reserve one sequence per
OST).<br>
(This allows the objects to potentially migrate
to<br>
different OSTs). The OSTs then internally must
map the<br>
FID to a local object id. Note this allows
OST-local<br>
precreation pools, getting the MDT out of the<br>
precreate/orphan cleanup business and potentially<br>
improving create speeds, and also facilitates
"create on<br>
write" semantics. The FID can be assigned during
the<br>
first access to OST object.</span></div>
</div>
</div>
</blockquote>
<br>
I am not sure I follow your idea here. You mean the OST
needs internally map MDT FID(added in OST number) to object
id (or inode ino) ?</div>
</blockquote>
yes.</div>
<div><br>
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000"> So there are no real
OST FID?</div>
</blockquote>
I suppose -- this is just a mapping of the MDT fid to the local
OST object id, via a local lookup on the OST. There would be
something like the OI to do this mapping.</div>
<div><br>
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000">But you also said "<span
style="font-size: 11pt; font-family: Arial; color: rgb(0,
0, 0); background-color: transparent; font-weight: normal;
font-style: normal; font-variant: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;"></span><span
style="font-size: 11pt; font-family: Arial; color: rgb(0,
0, 0); background-color: transparent; font-weight: normal;
font-style: normal; font-variant: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;">The
FID can be<br>
assigned during the first access to OST object.", Could
you please<br>
explain more here? </span><br>
</div>
</blockquote>
<div><br>
</div>
<div>Since the FID -> Objid mapping is performed locally, it
doesn't need to be assigned until the first write. This is
not integral to the design, just a side effect.</div>
</div>
</blockquote>
<br>
Ah, you mean the object ID can be assigned during the first access,
instead of FID? This is indeed an interesting idea, and do not need
extra space. But this may add some limits of the future. (what if we
decides to store some small file data in MDT directly?) And also it
will add another difference between MDT and OST, probably it
conflicts with the efforts of unifying MDT and OST? I still prefer
to have real OST FID, i.e. every object has its own identification
in the cluster. Please correct me if I miss the point of your
suggestion. <br>
<br>
Thanks<br>
Wangdi<br>
<br>
<blockquote
cite="mid:73AED5C780AE05478241DB067651A92102378850@XYUS-EX22.xyus.xyratex.com"
type="cite">
<div><br>
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000"> <br>
<br>
<blockquote
cite="mid:73AED5C780AE05478241DB067651A9210237883C@XYUS-EX22.xyus.xyratex.com"
type="cite">
<div>
<div style="background-color: transparent;">
<div style="font-family: Times; margin-left: 36pt;
margin-top: 0pt; margin-bottom: 0pt;"><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">The
big<br>
problem here is that FID>OBJID ( or better<br>
FID->inode id ) translation is absent from the
OSTs<br>
today. See </span><a moz-do-not-send="true"
href="http://wiki.lustre.org/images/e/e9/SC09-FID-on-OST.pdf"><span
style="font-size: 11pt; font-family: Arial;
color: rgb(0, 0, 153); background-color:
transparent; font-weight: normal; font-style:
normal; font-variant: normal; text-decoration:
underline; vertical-align: baseline;
white-space: pre-wrap;">http://wiki.lustre.org/images/e/e9/SC09-FID-on-OST.pdf</span></a><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;"><br>
(what is the current state of this?) There is
also some<br>
work in this direction in the OST restructuring
work<br>
(“Orion” WC branch, ORI-300(?), scheduled for
Lustre 2.4).<br>
</span></div>
<div style="margin-left: 36pt; margin-top: 0pt;
margin-bottom: 0pt;"><font class="Apple-style-span"
face="Arial"><span class="Apple-style-span"
style="font-size: 15px; white-space: pre-wrap;"><br>
<br>
</span></font></div>
</div>
<div style="background-color: transparent;"><span
style="font-size: 11pt; font-family: Arial; color:
rgb(0, 0, 0); background-color: transparent;
font-weight: normal; font-style: normal;
font-variant: normal; text-decoration: none;
vertical-align: baseline; white-space: pre-wrap;">There's<br>
a few questions here, probably the first of which is
"</span><span class="Apple-style-span"
style="font-family: Arial; font-size: 15px;
white-space: pre-wrap;">is it worthwhile to<br>
spend effort on this, or is BZ4424 good enough?"
Then there<br>
is the question of object identification, where
FID-on-OST<br>
is more flexible, but also significantly more work
(and<br>
risk). Also, I thought I understood from the EOFS
Summit<br>
that WC also has a separate FID-on-OST project
(separate<br>
from Orion that is) -- can someone tell me the state
of<br>
that?</span></div>
</div>
</blockquote>
<br>
FID-on-OST is actually part of DNE(dirtribute name space)
phase I. It basically follows current fid client server
infrastructure.<br>
<br>
1. MDT is the fid client, which requests fid from the OST
and allocates fids for the object during pre-creation. <br>
2. OST is the fid server, which will allocate the FIDs to
MDTs and requests super fid sequence from fid control server
(root MDT).<br>
3. Similar as MDT FID, there will be OI to map FID to object
inside OST.<br>
</div>
</blockquote>
<div><br>
</div>
<div>To integrate with this, we would need to have a reserved
sequence on each OST that the MDT could assign FIDs from --</div>
<div>the MDT would need to use the same Object ID on all OSTs.
For DNE, there would need to be a reserved sequence per OST
per MDT.</div>
<div><br>
</div>
<br>
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000"> <br>
The code will be release with DNE sometime next year.<br>
<br>
Thanks<br>
WangDi<br>
<br>
<br>
</div>
</blockquote>
</div>
<pre>______________________________________________________________________
This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it.
Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses.
Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.
The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People's Republic of China and Xyratex Japan Limited registered in Japan.
______________________________________________________________________
</pre>
</blockquote>
<br>
</body>
</html>