[Lustre-discuss] Loop device performance

[Nikolay Kvetsinski]

Thanks mate, I`ll try it right away.


On Mon, Aug 26, 2013 at 9:11 PM, Dilger, Andreas
<andreas.dilger at intel.com>wrote:

> On 2013/08/25 6:39 AM, "Nikolay Kvetsinski" <nkvecinski at gmail.com> wrote:
>
> >Hello, I have a production script that do read operations to a lot of
> >small files. I read that one can gain performance boost with small files
> >by using a loop device on top of Lustre. So a created 500 GB file striped
> >across all of my OSTs(which
> > are 8). I formatted the file with ext2 fs, and mounted it on a client.
> >Just for the sake of testing a simple bash script finds all files with a
> >given file type and cat the first 10 lines in /dev/null.
> >
> >
> >When I run the script on the Lustre cluster I get :
> >
> >
> >time sh test.sh
> >
> >
> >real    1m16.804s
> >user    0m2.539s
> >sys     0m5.363s
> >
> >
> >
> >If I immediately re-run the script the time is :
> >
> >
> >real    0m12.158s
> >user    0m2.218s
> >sys     0m5.430s
> >
> >
> >
> >
> >There are 5406 files that meet the filetype criteria.
> >
> >
> >When I run the script on the mounted loop device I get :
> >
> >
> >real    2m30.177s
> >user    0m2.290s
> >sys     0m4.880s
> >
> >And immediate re-run gives me :
> >
> >real    0m7.810s
> >user    0m2.187s
> >sys     0m5.360s
> >
> >
> >I`m usig lustre-2.4.0-2.6.32_358.6.2.el6_lustre.g230b174.x86_64_gd3f91c4.
> >Also
> >set all of "small files" optimizations like, no striping for the dirs
> >containing the small files, max_dirty_mb=256, max_rpcs_in_flight=32,
> >staahead=8192 and lnet.debug=0.
> >Is it normal to get two times slower access times with the mounted loop
> >device ??
>
> It depends on how the loop device is doing IO on the underlying objects.
> It may
> be that ext2 isn't the best filesystem for this.  You could try formatting
> it with:
>
>      mke2fs -t ext4 -O ^journal {device}
>
> which will enable the flex_bg,extents and other ext4 features but disables
> the
> journal (which I assume you don't need because you are formatting as ext2
> originally).  You should also mount with "-t ext4".
>
> The flex_bg and mballoc features of ext4 may help improve the IO going to
> the
> back-end storage and improve the performance when running over loop
> devices.
>
> Cheers, Andreas
> --
> Andreas Dilger
>
> Lustre Software Architect
> Intel High Performance Data Division
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130827/98db96af/attachment.htm>