[lustre-discuss] Benchmarking Lustre, reduce caching
Vicker, Darby J. (JSC-EG111)[Jacobs Technology, Inc.]
darby.vicker-1 at nasa.gov
Wed May 19 13:10:45 PDT 2021
I'd recommend using the io500 benchmark. This runs both bandwidth and metadata tests and has checks to run prevent caching from tainting the results (like forcing it to run for a specified amount of time). I've found it useful for benchmarking all of our file systems (lustre, NFS, local, etc.) and is very useful for comparing performance.
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of Athinagoras Skiadopoulos via lustre-discuss <lustre-discuss at lists.lustre.org>
Reply-To: Athinagoras Skiadopoulos <askiad at stanford.edu>
Date: Wednesday, May 19, 2021 at 1:36 PM
To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
Subject: [EXTERNAL] [lustre-discuss] Benchmarking Lustre, reduce caching
We are using Lustre in our cluster as an external service and
would like to measure the write/read performance we can achieve.
Right now, we are creating a number of files, and iteritavely:
- pick a file to open
- write/read it in 1MB chunks (block size)
- close it
Each of our servers has a 3000Mbps bandwidth,
but we are often reporting higher read/write throughput.
There should be some caching involved.
What would be a good way to alleviate this?
Thank you in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the lustre-discuss