I have 1 OST Per OSS.<br>I have set the stripe to Lustre default. Should I still change the stripe count to 1?<br>Yes, the reads are sequential.<br><br>1. Will increasing / decreasing the number of max_rpcs_in_flight help?<br>
It is set to 32 now.<br><br>2. Just for experimenting, would it speed up listing if 1 OSS served 2 OSTs, thereby reducing the number of OSS and RPCs?<br><br>Regards,<br><br><br>Indivar Nair<br><br><div class="gmail_quote">
On Wed, Sep 7, 2011 at 1:28 AM, Adrian Ulrich <span dir="ltr"><<a href="mailto:adrian@blinkenlights.ch">adrian@blinkenlights.ch</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im"><br>
> While normal file access works fine, the directory listing is extremely<br>
> slow.<br>
> Depending on the number of files in a directory, the listing takes around 5<br>
> - 15 secs.<br>
><br>
> I tried 'ls --color=none' and it worked fine; listed the contents<br>
> immediately.<br>
<br>
</div>That's because 'ls --color=always|auto' does an lstat() of each file (--color=none doesn't) which causes Lustre to send:<br>
<br>
- 1 RPC to the MDS per file<br>
- 1 RPC (per file) to EACH OSS where the file is stored to get the filesize<br>
<br>
Some time ago i've created a patch to speed up 'ls' while keeping (most) of the colors<br>
(<a href="https://github.com/adrian-bl/patchwork/blob/master/coreutils/ls/ls-lustre.diff" target="_blank">https://github.com/adrian-bl/patchwork/blob/master/coreutils/ls/ls-lustre.diff</a>)<br>
<br>
But patching samba will not be possible in your case as it really needs the information returned by stat().<br>
<div class="im"><br>
<br>
> Double clicking on directory takes a long long time to display.<br>
<br>
</div>Attach `strace' to samba: It will probably be busy doing lstat() which is a 'slow' operation on Lustre in any case.<br>
<div class="im"><br>
<br>
<br>
> The cluster consist of -<br>
> - two DRBD Mirrored MDS Servers (Dell R610s) with 10K RPM disks<br>
> - four OSS Nodes (2 Node Cluster (Dell R710s) with a common storage (Dell<br>
> MD3200))<br>
<br>
</div>How many OSTs do you have per OSS?<br>
What's your stripe setting? Setting the stripe to 1 could give you a huge speedup (without affecting normal I/O as i assume that the 9MB files are read/written sequentially)<br>
<br>
<br>
Regards,<br>
Adrian<br>
<br>
<br>
--<br>
RFC 1925:<br>
(11) Every old idea will be proposed again with a different name and<br>
a different presentation, regardless of whether it works.<br>
<br>
_______________________________________________<br>
Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
</blockquote></div><br>