[lustre-discuss] POC Performance

[Bill Carlson]

Note the tests were run on a 20 cpu, 1TB (not GB) ram, also serving as 
the only OSS for the filesystem.

I've noted a key item, sysbench tests were with 16k blocks. Changing to 
1MB was much much better, only about 10% slower than raw disk.

Time to move to the next phase. :)

-- 

Bill Carlson

Anything is possible, given Time and Money.

On 4/8/19 3:01 PM, Riccardo Veraldi wrote:
> Are you testing this in a virtual machine environment ?
>
> if you are aiming for Lustre performacne you should not run virtual
> machines especially on the OSS side.
> Also how many clients are you using for reading/writing ? to get
> performance out of lustre or any other parallel filesystem, you need to
> stripe and read/write in paralle from different clients. Also if you run
> Lustre/ZFS you need a good amount of RAM and 8GB on the OSS side sounds
> like too few for me.
>
> In my environment I have 8x OSS each one with 128GB RAM and a raidz with
> 4x Micron NVMe 9200  disks.
> Each OSS has a Infiniband EDR connection. In my test I use 32 threads
> per client writing from 8 clients at the same time on the Lustre
> filesystem and
> I can saturate the NVMe SSD disks performance. I get almost 80GB/s write
> performance and 90GB/s read performance over Infiniband.
>
>
>
> On 4/8/19 12:46 PM, Bill Carlson wrote:
>> Hello,
>>
>> I've been chasing a proof of concept for Lustre, so far performance
>> tests are not promising.
>> Basic setup:
>>
>> MGS/MDT: VM, 4 cpu, 8GB ram
>> OSS #1: VM, 16 cpu, 8GB ram
>> OSS #1: hardware, 20 cpu, 1TB ram
>>
>> I've been using sybench fileio for tests, 16k on 50GB over 5 minutes.
>>
>> Basic test results, performed on OSS with mounted FS:
>> Base ext4 SSD, OSS #2:
>> Sequential write: 1 GB/s
>> Random r/w: 551 MB/s read, 367 MB/s write
>>
>> ZFS dataset SSD, OSS #2:
>> Sequential write:  397 MB/s
>> Randow r/w: 109 MB/s read, 73 MB/s write
>>
>> About 5 times slower. Expected?
>>
>>
>> ZFS OST SSD, OSS #2:
>> Sequential write: 9 MB/s
>> Randow r/w: 18MB/s read, 12.5MB/s write
>>
>> Over 30-110 times slower than basic disk, that just doesn't seem right.
>> I also tried ldiskfs, not much difference.
>>
>> I tried various changes, ZFS compression on, atime off, xattr sa.
>>
>> Watching the system via atop during a 5 minute OST test, disks are not
>> 100% busy and CPU is mostly idle. Network is all lo.
>>
>> What am I missing? I assumed random r/w would be pretty slow, but not
>> sequential.
>>
>> Thanks,
>>
>