[Lustre-discuss] Loop device performance
nkvecinski at gmail.com
Sun Aug 25 05:39:38 PDT 2013
Hello, I have a production script that do read operations to a lot of small
files. I read that one can gain performance boost with small files by using
a loop device on top of Lustre. So a created 500 GB file striped across all
of my OSTs(which are 8). I formatted the file with ext2 fs, and mounted it
on a client. Just for the sake of testing a simple bash script finds all
files with a given file type and cat the first 10 lines in /dev/null.
When I run the script on the Lustre cluster I get :
time sh test.sh
If I immediately re-run the script the time is :
There are 5406 files that meet the filetype criteria.
When I run the script on the mounted loop device I get :
And immediate re-run gives me :
I`m usig lustre-2.4.0-2.6.32_358.6.2.el6_lustre.g230b174.x86_64_gd3f91c4.
Also set all of "small files" optimizations like, no striping for the dirs
containing the small files, max_dirty_mb=256, max_rpcs_in_flight=32,
staahead=8192 and lnet.debug=0.
Is it normal to get two times slower access times with the mounted loop
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lustre-discuss