[Lustre-discuss] Slow Copy (Small Files) 1 RPC In Flight?

[Andrew Mast]

Hello, I am new to Lustre and wanted to run a small simple small copy test
between 2 virtual machines from MDT/OST server to client's local disk.

I realize small file performance is never fast, but this seems particularly
slow considering the data is all buffered in memory with little to no disk
activity.

Setup Info
Version is 2.4.50
Average file size is small. < 10KB
The amount of data being copied is about 250MB.
The VMs are on separate hosts.

Performance
7 minutes over a gigabit network.
NFS takes only 3 minutes.

Observations
iostat on the OST/MDT is usually 0% during the copy. Assuming all buffered.
Additional network traffic is minimal.
CPU load on the VMs is 15-20% during copy.

RPC stats on the client shows only 1 RPC in flight at a time. max inflight
is set to 64. Is that expected behavior for a copy?

Here is a snapshot of rpc_stats early during the copy:

   read write pages per rpc rpcs % cum % | rpcs % cum % 1: 1653 90 90 | 0 0
0 2: 164 8 98 | 0 0 0 4: 7 0 99 | 0 0 0 8: 3 0 99 | 0 0 0 16: 3 0 99 | 0 0
0 32: 5 0 99 | 0 0 0 64: 0 0 99 | 0 0 0 128: 1 0 100 | 0 0 0 read write
rpcs in flight rpcs % cum % | rpcs % cum % 0: 0 0 0 | 0 0 0 1: 1836 100 100
| 0 0 0 read write offset rpcs % cum % | rpcs % cum % 0: 1836 100 100 | 0 0
0

As I am new, any suggestions for what to look for or improve would be
greatly appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20130621/886e87de/attachment.htm>