[Lustre-discuss] Loop device performance

[Nikolay Kvetsinski]

Hello, I have a production script that do read operations to a lot of small
files. I read that one can gain performance boost with small files by using
a loop device on top of Lustre. So a created 500 GB file striped across all
of my OSTs(which are 8). I formatted the file with ext2 fs, and mounted it
on a client. Just for the sake of testing a simple bash script finds all
files with a given file type and cat the first 10 lines in /dev/null.

When I run the script on the Lustre cluster I get :

time sh test.sh

real    1m16.804s
user    0m2.539s
sys     0m5.363s

If I immediately re-run the script the time is :

real    0m12.158s
user    0m2.218s
sys     0m5.430s


There are 5406 files that meet the filetype criteria.

When I run the script on the mounted loop device I get :

real    2m30.177s
user    0m2.290s
sys     0m4.880s

And immediate re-run gives me :

real    0m7.810s
user    0m2.187s
sys     0m5.360s

I`m usig lustre-2.4.0-2.6.32_358.6.2.el6_lustre.g230b174.x86_64_gd3f91c4.
Also set all of "small files" optimizations like, no striping for the dirs
containing the small files, max_dirty_mb=256, max_rpcs_in_flight=32,
staahead=8192 and lnet.debug=0.
Is it normal to get two times slower access times with the mounted loop
device ??
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130825/12e9f7ff/attachment.htm>