<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hello,</p>
<p>I have a lustre cluster on rhel7, 6 OSS each of them has 3 OSTs
and 1 MDS.</p>
<p>I am using lustre on ZFS.<br>
</p>
While write performances are excellent also on smaller files, I find
there is a drop down in performance<br>
on reading 20KB files. Performance can go as low as 200MB/sec or
even less.<br>
I am talking about random reads and random stride reads.<br>
I did the following to try to improve things:
<ul>
<li>disabled lnet debug messages:</li>
<ul>
<li>sysctl -w lnet.debug=0 <br>
</li>
</ul>
<li>increased dirty cache</li>
<ul>
<li>lctl set_param osc.lutrefs\*.max_dirty_mb=256</li>
</ul>
<li>increased number of RPC in flight</li>
<ul>
<li>for i in `ls
/proc/fs/lustre/osc/lustrefs-OST00*/max_rpcs_in_flight`; do
echo 32 > $i; done</li>
</ul>
</ul>
it did not improve reading 20KB file performances.<br>
I have to say in advance I did not set up any striping because I
will have no more than 6 concurrent reads and writes,<br>
so striping is not that much important for me.<br>
Here the problem is that one single random read of a 20KB file is
around 190MB/s and this is really disappointing.<br>
I am open to any suggestion.<br>
I made sure it is not a ZFS problem, on the OSSs ZFS is performing
like a charm locally.
<p>thank you</p>
<p><br>
</p>
<p>Riccardo</p>
<p><br>
</p>
</body>
</html>