<div dir="ltr">Hello, I am new to Lustre and wanted to run a small simple small copy test between 2 virtual machines from MDT/OST server to client's local disk.<div><br></div><div>I realize small file performance is never fast, but this seems particularly slow considering the data is all buffered in memory with little to no disk activity.<div>
<br></div><div style>Setup Info</div><div>Version is 2.4.50<br><div style>Average file size is small. < 10KB</div><div>The amount of data being copied is about 250MB.</div><div style><div>The VMs are on separate hosts.<br>
</div><div><br></div><div style>Performance</div></div><div style>7 minutes over a gigabit network. </div><div style>NFS takes only 3 minutes.</div><div style><br></div><div style>Observations</div><div style>iostat on the OST/MDT is usually 0% during the copy. Assuming all buffered.<br>
</div><div style>Additional network traffic is minimal. </div><div style>CPU load on the VMs is 15-20% during copy.</div><div style><br></div><div style>RPC stats on the client shows only 1 RPC in flight at a time. max inflight is set to 64. Is that expected behavior for a copy?</div>
<div style><br></div><div style>Here is a snapshot of rpc_stats early during the copy:</div><div style><br></div><div style><div> <span style="white-space:pre">read write
pages per rpc rpcs % cum % | rpcs % cum %
1: 1653 90 90 | 0 0 0
2: 164 8 98 | 0 0 0
4: 7 0 99 | 0 0 0
8: 3 0 99 | 0 0 0
16: 3 0 99 | 0 0 0
32: 5 0 99 | 0 0 0
64: 0 0 99 | 0 0 0
128: 1 0 100 | 0 0 0
read write
rpcs in flight rpcs % cum % | rpcs % cum %
0: 0 0 0 | 0 0 0
1: 1836 100 100 | 0 0 0
read write
offset rpcs % cum % | rpcs % cum %
0: 1836 100 100 | 0 0 0
</span></div><div><span style="white-space:pre"><br></span></div><div style><span style="white-space:pre">As I am new, any suggestions for what to look for or improve would be greatly appreciated.</span></div></div></div>
</div></div>