[lustre-discuss] Why reads are slower than writes on lustre file system?

Ben Evans beevans at whamcloud.com
Tue Sep 28 09:50:14 PDT 2021


On the second experiment, you’re writing a total of 1000MB and reading 100MB.  It could simply be that you’re not putting enough load on the system for long enough to get full performance.

-Ben Evans

From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Colin Faber via lustre-discuss <lustre-discuss at lists.lustre.org>
Reply-To: Colin Faber <cfaber at gmail.com>
Date: Monday, September 27, 2021 at 9:50 PM
To: Nagmat Nazarov <nagmat at nevada.unr.edu>
Cc: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: Re: [lustre-discuss] Why reads are slower than writes on lustre file system?

Depending on your IO workloads reads can be slower than writes in cases where the writes may be sequential and optimized for the backing storage hardware, and the reads random, or semi-random. This also can be greatly affected by block allocator efficiency. Typically on well tuned modern lustre file systems reads and writes are very similar (if not better for reads as Paf mentions).

On Mon, Sep 27, 2021 at 5:30 PM Nagmat Nazarov <nagmat at nevada.unr.edu<mailto:nagmat at nevada.unr.edu>> wrote:
My backing file system is ldiskfs.
The storage hardware is HDD I guess(Since I am using emulab cloud storage)
I am doing buffered I/O.

On Mon, Sep 27, 2021 at 4:20 PM Patrick Farrell <pfarrell at ddn.com<mailto:pfarrell at ddn.com>> wrote:
It does depend on your storage hardware, but modern Lustre software is generally the same or faster for reads.

What is your backing file system - ldiskfs or ZFS?  And what is the storage hardware?  And are you doing direct or buffered I/O?
________________________________
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org>> on behalf of Nagmat Nazarov <nagmat at nevada.unr.edu<mailto:nagmat at nevada.unr.edu>>
Sent: Monday, September 27, 2021 5:57 PM
To: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org> <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>>
Subject: [lustre-discuss] Why reads are slower than writes on lustre file system?

Dear Engineers,

I have started working on a lustre file system. I have done couple of experiments so far:
On the first experiment I am writing 100 files each 10MB and the batch size is 4K. I got 107MB/s bandwidth.

On the second experiment I am (writing 10 files each 10MB and reading 1 10 MB file back) for 10 times.  Here also the average bandwidth is 106MB/s while  read average bandwidth is 59MB/s which is very weird I guess?

My question is, generally on ext4 file systems read average bandwidth is faster that write average bandwidth, why is it 2 times slower on lustre file system. My ethernet speed is 1Gb/s.

Kind regards
Nagmat Nazarov


_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20210928/833fb23e/attachment.html>


More information about the lustre-discuss mailing list