[Lustre-devel] Large readdir RPCs project

[Jeremy Filizetti]

I've put together a small patch to modify ll_dir_readpage and mdc_readpage
to read extra pages (if available) with each RPC.  It is posted under the
bug (https://bugzilla.lustre.org/show_bug.cgi?id=17833).  I see you we're
the original assignee when you were at Sun/Oracle.

Jeremy

2010/9/29 Fan Yong <yong.fan at whamcloud.com>

>   On 9/30/10 3:01 AM, Jeremy Filizetti wrote:
>
>
>> On the other hand, Large readdir RPCs is basic of another metadata read
>> performance improvement features - "readdir+", which is quite useful for "ls
>> -l" operation (for large directory), and reduce lookup/getattr RPC as much
>> as possible. In such feature, MDS will pack more dir-item's attribute (not
>> only name/ino as does currently by readdir, but also mode/owner, and etc)
>> information back to client-side in "readdir+" RPC. That means the dir-item
>> count in one "readdir+" page is less than in the traditional readdir page,
>> then more pages to be sent back to client. If without "Large readdir RPCs",
>> the advantage of "readdir+" will be discounted.
>>
>
> I'd be interested in working this as well but probably as a separate
> effort since SOM isn't in 1.8 and that's my main focus.  In our testing, SOM
> had significant benefits over the WAN and I'd expect even better from
> readdir+.  I have tried Oleg's patch for asynchronous ll_glimpse_size but
> oddly I've seen someone erradic performance where at times it was worse then
> statahead and synchronous ll_glimpse_size.
>
> Jeremy
>
>
> Yes, SOM is another important feature for metadata reading performance
> improvement by bypassing the glimpse RPC between client and OSS. Engineers
> from Lustre Group worked for that for some time, hope can be released soon.
>
> As for the asynchronous ll_glimpse_size maybe cause bad performance
> occasionally, one possible reason is that: glimpse RPC maybe not obtain
> extent lock(s) if some others in using such lock(s), so the file size
> information obtained by asynchronous glimpse is invalid when it is really
> used later, means the caller ("stat") has to send synchronous glimpse again.
> Anyway, I did not study such patch, so it is just a guess.
>
>
> Cheers,
> Nasf
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20101013/df19abc3/attachment.htm>