<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
On Jun 28, 2021, at 16:58, Vinayak.Kamath via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a>> wrote:<br class="">
<div>
<blockquote type="cite" class=""><br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Greetings,<o:p class=""></o:p></span></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Our team in the process of setting up an HPC system. We\u2019re evaluating several distributed file systems(DFS), including Lustre. </span><span style="font-size: 11pt;" class="">This is new territory for us and we\u2019ve been learning a lot
as we go.</span></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class=""> <o:p class=""></o:p></span></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Those of you who have been breathing file system for years \u2013 are fio and ior good tools to benchmark a DFS? What are some common pitfalls to avoid while comparing filesystems?<o:p class=""></o:p></span></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">I suppose both these questions are complex topics worth writing papers and books on. Are there any good references that you can suggest?</span></div>
</div>
</div>
</blockquote>
<br class="">
</div>
<div>It depends on what your target application(s) are (no pun intended). Ideally, you would benchmark your actual application.</div>
<div><br class="">
</div>
<div>For HPC, IOR is most commonly used for bandwidth, mdtest for file operations. These two are sometimes combined into an "aggregate" benchmark called IO500 that aggregates multiple different kinds of workloads (streaming, interleaved of large and small
files, metadata create, lookup, unlink) (<a href="https://io500.org/" class="">https://io500.org/</a>, caveat I'm on the board for IO500 as well).</div>
<div><br class="">
</div>
<div>Fio is mostly used for small block read/write IOPS measurements, and testing non-POSIX IO interfaces like libaio, io_uring, mmap, etc.</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</body>
</html>