[lustre-discuss] poor performance on reading small files
Riccardo Veraldi
Riccardo.Veraldi at cnaf.infn.it
Thu Jul 28 19:19:20 PDT 2016
Hello,
I have a lustre cluster on rhel7, 6 OSS each of them has 3 OSTs and 1 MDS.
I am using lustre on ZFS.
While write performances are excellent also on smaller files, I find
there is a drop down in performance
on reading 20KB files. Performance can go as low as 200MB/sec or even less.
I am talking about random reads and random stride reads.
I did the following to try to improve things:
* disabled lnet debug messages:
o sysctl -w lnet.debug=0
* increased dirty cache
o lctl set_param osc.lutrefs\*.max_dirty_mb=256
* increased number of RPC in flight
o for i in `ls
/proc/fs/lustre/osc/lustrefs-OST00*/max_rpcs_in_flight`; do echo
32 > $i; done
it did not improve reading 20KB file performances.
I have to say in advance I did not set up any striping because I will
have no more than 6 concurrent reads and writes,
so striping is not that much important for me.
Here the problem is that one single random read of a 20KB file is
around 190MB/s and this is really disappointing.
I am open to any suggestion.
I made sure it is not a ZFS problem, on the OSSs ZFS is performing like
a charm locally.
thank you
Riccardo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160728/3f81ea6a/attachment.htm>
More information about the lustre-discuss
mailing list