<font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">Hi Andreas,</font></font></font><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br></font></font></font></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">Thanks for your e-mail.</font></font></font></div><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br>
</font></font></font></div><div><font color="#3333ff"><font size="2"><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 12.5px; border-collapse: collapse; "><font color="#3333ff"><font size="2"><div><font class="Apple-style-span" face="georgia, serif">We are trying to do following things.Please let me know if things are not clear :-</font></div>
</font></font><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br></font></font></font></div><div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">Say we have 2 client C1 and C2 and a MDS .</font></span><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">Say C1 and C2 share a file.</font></span></div>
<div><font class="Apple-style-span" face="georgia, serif"><span style="color: rgb(51, 51, 255); "></span></font><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">1) When a client C1 performs a open/create kind of request to the MDS we want to follow the normal path which Lustre performs.</font></span></div>
<div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">2) Now say C2 tries to open the same file which was opened by C1.</font></span></div><div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">3) At the MDS end we maintain some data structure to scan and see if the file was already opened by some Client(in this case C1 has opened this file).</font></span></div>
<div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">4) If MDS finds that some client(C1 here) has already opened the file then it send the new client(C2 here) with some information about the client which has initially opened the file.</font></span></div>
<div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">5) Once C2 gets the information its upto C2 to take further actions.</font></span></div><div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">6) By this process we can save the time spent in the locking mechanism for C2.Basically we aim to by-pass the locking scheme of Lustre for the files already opened by some client by maintaining some kind of data structure.</font></span></div>
<div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif"><br></font></span></div><div><span style="color: rgb(51, 51, 255); "><font class="Apple-style-span" face="georgia, serif">Please let us know your thoughts on the above approach.Is this a feasible design moving ahead can we see any complications ?</font></span></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br></font></font></font></div><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">So considering the problem statement I need a way for C2 to extract the information from the data structure maintained at MDS.In order to do that , C2 will send a request with intent = create|open which will be a LDLM_ENQUEUE RPC.I need to modify this RPC such that :-</font></font></font></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">1) I can enclose some additional buffer whose size is known to me .</font></font></font></div><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">2) When we pack the reply at the MDS side we should be able to include this buffer in the reply message .</font></font></font></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">3) At the client side we should be able to extract the information from the reply message about the buffer.</font></font></font></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br></font></font></font></div><div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif">As of now , I need help in above three steps.</font></font></font></div>
<div><font color="#3333ff"><font size="2"><font class="Apple-style-span" face="georgia, serif"><br clear="all"></font><span class="Apple-style-span" style="border-collapse: separate; "><font class="Apple-style-span" face="georgia, serif">Thanks,</font></span></font></font></div>
</span></font></font><font style="color:rgb(51, 51, 255)" size="2" color="#3333ff"><font class="Apple-style-span" face="georgia, serif">Vilobh<br></font>
</font><i><font style="color:rgb(51, 51, 255)" size="2"><font class="Apple-style-span" face="georgia, serif">Graduate Research Associate<br>Department of Computer Science<br>The Ohio State University Columbus Ohio</font></font></i><font class="Apple-style-span" face="'times new roman', serif"><br>
</font>
<br><br><div class="gmail_quote">On Tue, Oct 19, 2010 at 6:53 PM, Andreas Dilger <span dir="ltr"><<a href="mailto:andreas.dilger@oracle.com">andreas.dilger@oracle.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On 2010-10-19, at 14:28, Vilobh Meshram wrote:<br>
> From my exploration it seems like for create/open kind of request LDLM_ENQUEUE is the RPC through which the client talks to MDS.Please confirm on this.<br>
><br>
> Since I could figure out that LDLM_ENQUEUE is the only RPC to interface with MDS I am planning to send the LDLM_ENQUEUE RPC with some additonal buffer from the client to the MDS so that based on some specific condition the MDS can fill the information in the buffer sent from the client.<br>
<br>
</div>This isn't correct. LDLM_ENQUEUE is used for enqueueing locks. It just happens that when Lustre wants to create a new file it enqueues a lock on the parent directory with the "intent" to create a new file. The MDS currently always replies "you cannot have the lock for the directory, I created the requested file for you". Similarly, when the client is getting attributes on a file, it needs a lock on that file in order to cache the attributes, and to save RPCs the attributes are returned with the lock.<br>
<div class="im"><br>
> I have made some modifications to the code for the LDLM_ENQUEUE RPC but I am getting kernel panics.Can someone please help me and suggest me what is a good way to tackle this problem.I am using Lustre 1.8.1.1 and I cannot upgrade to Lustre 2.0.<br>
<br>
</div>It would REALLY be a lot easier to have this discussion with you if you actually told us what it is you are working on. Not only could we focus on the higher-level issue that you are trying to solve (instead of possibly wasting a lot of time focussing in a small issue that may in fact be completely irrelevant), but with many ideas related to Lustre it has probably already been discussed at length by the Lustre developers sometime over the past 8 years that we've been working on it. I suspect that the readership of this list could probably give you a lot of assistance with whatever you are working on, if you will only tell us what it actually is you are trying to do.<br>
<div class="im"><br>
> On Mon, Oct 18, 2010 at 7:33 PM, Vilobh Meshram <<a href="mailto:vilobh.meshram@gmail.com">vilobh.meshram@gmail.com</a>> wrote:<br>
</div><div class="im">>> Out of the many RPC's used in Lustre seems like LDLM_ENQUEUE is the most frequently used RPC to communicate between the client and the MDS.I have few queries regarding the same :-<br>
>><br>
>> 1) Is LDLM_ENQUEUE the only interface(RPC here) for CREATE/OPEN kind of request ; through which the client can interact with the MDS ?<br>
>><br>
>> I tried couple of experiments and found out that LDLM_ENQUEUE comes into picture while mounting the FS as well as when we do a lookup,create or open a file.I was expecting the MDS_REINT RPC to get invoked in case of a CREATE/OPEN request via mdc_create() but it seems like Lustre invokes LDLM_ENQEUE even for CREATE/OPEN( by packing the intent related data).<br>
>> Please correct me if I am wrong.<br>
>><br>
>> 2) In which cases (which system calls) does the MDS_REINT RPC will get invoked ?<br>
<br>
<br>
</div>Cheers, Andreas<br>
<font color="#888888">--<br>
Andreas Dilger<br>
Lustre Technical Lead<br>
Oracle Corporation Canada Inc.<br>
<br>
</font></blockquote></div><br></div>