[lustre-discuss] POC Performance

[Bill Carlson]

Hello,

I've been chasing a proof of concept for Lustre, so far performance 
tests are not promising.
Basic setup:

MGS/MDT: VM, 4 cpu, 8GB ram
OSS #1: VM, 16 cpu, 8GB ram
OSS #1: hardware, 20 cpu, 1TB ram

I've been using sybench fileio for tests, 16k on 50GB over 5 minutes.

Basic test results, performed on OSS with mounted FS:
Base ext4 SSD, OSS #2:
Sequential write: 1 GB/s
Random r/w: 551 MB/s read, 367 MB/s write

ZFS dataset SSD, OSS #2:
Sequential write:  397 MB/s
Randow r/w: 109 MB/s read, 73 MB/s write

About 5 times slower. Expected?


ZFS OST SSD, OSS #2:
Sequential write: 9 MB/s
Randow r/w: 18MB/s read, 12.5MB/s write

Over 30-110 times slower than basic disk, that just doesn't seem right.
I also tried ldiskfs, not much difference.

I tried various changes, ZFS compression on, atime off, xattr sa.

Watching the system via atop during a 5 minute OST test, disks are not 
100% busy and CPU is mostly idle. Network is all lo.

What am I missing? I assumed random r/w would be pretty slow, but not 
sequential.

Thanks,

-- 

Bill Carlson

Anything is possible, given Time and Money.