<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
NVidia seems to think so:
<div><br>
</div>
<div><a href="https://wiki.lustre.org/images/3/30/LUG2021-Accelerating_AI_at_Scale-Bernauer-Kashinkunti.pdf">https://wiki.lustre.org/images/3/30/LUG2021-Accelerating_AI_at_Scale-Bernauer-Kashinkunti.pdf</a><br>
<br>
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Jun 28, 2021, at 17:43, Vinayak.Kamath <Vinayak.Kamath@target.com> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">\ufeff
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal">Thanks for the prompt response, Andreas.<o:p></o:p></p>
<p class="MsoNormal">Is Lustre a good choice for a non-volatile \u201ccache\u201d of ML training datasets? <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks,<o:p></o:p></p>
<p class="MsoNormal">Vinayak<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">Andreas Dilger <adilger@whamcloud.com><br>
<b>Date: </b>Monday, June 28, 2021 at 4:23 PM<br>
<b>To: </b>"Vinayak.Kamath" <Vinayak.Kamath@target.com><br>
<b>Cc: </b>"lustre-discuss@lists.lustre.org" <lustre-discuss@lists.lustre.org><br>
<b>Subject: </b>[EXTERNAL] Re: [lustre-discuss] What's your favorite distributed filesystem benchmark?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">On Jun 28, 2021, at 16:58, Vinayak.Kamath via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>> wrote:<o:p></o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">Greetings,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Our team in the process of setting up an HPC system. We\u2019re evaluating several distributed file systems(DFS), including Lustre. This is new territory for us and we\u2019ve been learning a lot as we go.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Those of you who have been breathing file system for years \u2013 are fio and ior good tools to benchmark a DFS? What are some common pitfalls to avoid while comparing filesystems?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I suppose both these questions are complex topics worth writing papers and books on. Are there any good references that you can suggest?<o:p></o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">It depends on what your target application(s) are (no pun intended). Ideally, you would benchmark your actual application.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For HPC, IOR is most commonly used for bandwidth, mdtest for file operations. These two are sometimes combined into an "aggregate" benchmark called IO500 that aggregates multiple different kinds of workloads (streaming, interleaved of
large and small files, metadata create, lookup, unlink) (<a href="https://io500.org/">https://io500.org/</a>, caveat I'm on the board for IO500 as well).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Fio is mostly used for small block read/write IOPS measurements, and testing non-POSIX IO interfaces like libaio, io_uring, mmap, etc.<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">Cheers, Andreas<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">--<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Andreas Dilger<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Lustre Principal Architect<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Whamcloud<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
</body>
</html>