Hi Wangdi<br><br>I understand the process that you said and give me a clearly example. So, In the lustre system internal, there is no "physical" file joining process ? My lustre client works as web server, serving download. In this case, the "physical" data joining process are on our web client , isn't it ? <br>
<br>One more thing, just need a confirmation, in the joining file process, OSSs doesn't have any role, right ? They just handle I/O action? <br><br>Sorry if i'm rising such a stupid question :) <br><br>Many thanks <br>
<br><div class="gmail_quote">On Thu, Jan 14, 2010 at 10:55 AM, Tom.Wang <span dir="ltr"><<a href="mailto:Tom.Wang@sun.com">Tom.Wang@sun.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Lex wrote:<div class="im"><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
On Wed, Jan 13, 2010 at 5:40 PM, Tom.Wang <<a href="mailto:Tom.Wang@sun.com" target="_blank">Tom.Wang@sun.com</a> <mailto:<a href="mailto:Tom.Wang@sun.com" target="_blank">Tom.Wang@sun.com</a>>> wrote:<br>
<br>
Lex wrote:<br>
<br>
Hi guys<br>
<br>
I have some questions about file joining mechanism in lustre,<br>
so i post here and hope you can help me to clarify it :)<br>
<br>
As we know, lustre use the split writing and hierarchy<br>
striping to distribute file object ( subfile) to many osts in<br>
the cluster. When a client request one file ( over mds<br>
metadata ), these objects are retrieve from each ost to client.<br>
<br>
So, the question is: when and where these objects are joined<br>
together to become a real file ?<br>
<br>
File joining is actually joining the file stripe information.<br>
which is stored in MDS, so the joining actually happens in MDS,<br>
instead of OSS.<br>
<br>
Thank you for you fast reply, but i mean the physical joining file process, not just information or metadata. As i know, file ( data ) doesn't go over MDS, right ? Sorry if i'm misunderstanding you<br>
</blockquote></div>
The stripe information is the layout information of file data, which indicates where those data is really stored on OST. And the joining process is actually joining these layout information, so the two file_data looks like joined together. For example you have<br>
<br>
file_A size 8k stripe_A file_B size 16k stripe_B<br>
<br>
then the stripe information of joinAB will be sth like file extent<br>
<br>
[0, 8k] stripe_A ---------------> original file_A data on OST<br>
[8k, 24k] stripe_B --------------> original file_B data on OST<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Btw: Because of some reasons, joining file feature will be removed<br>
temporarily in lustre2.0.<br>
<br>
<br>
Could you please tell me exactly what the reasons are ?<br>
</blockquote></div>
On 2.0, MDS is completely restructured (you can see that on lustre roadmap) because of cluster MDS, so join file also needs to be re-implemented. But it is a low priority project, and we do not have enough resource to do that now. We might do that in some later version.<br>
<br>
Thanks<br>
Wangdi<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="im">
<br>
<br>
Thanks<br>
WangDi<br>
<br>
<br>
Just guess, are there joined together in OSSs? If it's true,<br>
how these OSSs connect together to join its objects together?<br>
( For example, assuming we have 2 OSSs, each OSS have 1 OST,<br>
file are striped across to 2 these OSTs. When a client request<br>
a file, i get to object in 2 different OST from 2 different<br>
OSSs, so how these objects can combined together? )<br>
<br>
Any help would be appreciated :)<br>
<br>
Many thanks<br>
<br>
<br>
<br>
------------------------------------------------------------------------<br>
<br>
_______________________________________________<br>
Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a><br></div>
<mailto:<a href="mailto:Lustre-discuss@lists.lustre.org" target="_blank">Lustre-discuss@lists.lustre.org</a>><div class="im"><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
<br>
<br>
<br>
</div></blockquote>
<br>
</blockquote></div><br>