<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<p>I'll assume here you're referring to MPI-IO, which is not really
a "feature" but a way to perform parallel I/O using the message
passing interface (MPI) stack. There can also be different
interpretations of what MPI-IO exactly means: many HPC
applications (and benchmarks such as IOR) use MPI and can write
out data in parallel using various I/O APIs. One of those APIs for
IOR is "mpiio", from the web page:</p>
<p> </p>
<div class="page" title="Page 19">
<div class="layoutArea">
<div class="column">
<pre><span style="font-size: 10.000000pt; font-family: 'NimbusRomNo9L'; font-weight: 500">-a S </span><span style="font-size: 10.000000pt; font-family: 'NimbusRomNo9L'">api – API for I/O [POSIX|MPIIO|HDF5|HDFS|S3|S3_EMC|NCMPI|RADOS]
In short, running parallel threads to write files will usually boost your IOPs to a point, though which API you choose should probably reflect what your real-life applications use.
Cameron
</span></pre>
</div>
<div class="column"> </div>
</div>
</div>
<div class="moz-cite-prefix">On 3/18/23 2:44 AM, 王烁斌 via
lustre-discuss wrote:<br>
</div>
<blockquote type="cite" cite="mid:728ca058.71.186f3acea5b.Coremail.w14767780617@163.com">
<div style="line-height:1.7;color:#000000;font-size:14px;font-family:Arial">
<div>Hi all,<br>
</div>
<div><br>
</div>
<div>This is my Lustre FS. </div>
<div>
<div>UUID 1K-blocks Used Available
Use% Mounted on</div>
<div>ltfs-MDT0000_UUID 307826072 36904 281574768
1% /mnt/lfs[MDT:0]</div>
<div>ltfs-MDT0001_UUID 307826072 36452 281575220
1% /mnt/lfs[MDT:1]</div>
<div>ltfs-MDT0002_UUID 307826072 36600 281575072
1% /mnt/lfs[MDT:2]</div>
<div>ltfs-MDT0003_UUID 307826072 36300 281575372
1% /mnt/lfs[MDT:3]</div>
<div>ltfs-OST0000_UUID 15962575136 1027740 15156068868
1% /mnt/lfs[OST:0]</div>
<div>ltfs-OST0001_UUID 15962575136 1027780 15156067516
1% /mnt/lfs[OST:1]</div>
<div>ltfs-OST0002_UUID 15962575136 1027772 15156074212
1% /mnt/lfs[OST:2]</div>
<div>ltfs-OST0003_UUID 15962575136 1027756 15156067860
1% /mnt/lfs[OST:3]</div>
<div>ltfs-OST0004_UUID 15962575136 1027728 15156058224
1% /mnt/lfs[OST:4]</div>
<div>ltfs-OST0005_UUID 15962575136 1027772 15156057668
1% /mnt/lfs[OST:5]</div>
<div>ltfs-OST0006_UUID 15962575136 1027768 15156058568
1% /mnt/lfs[OST:6]</div>
<div>ltfs-OST0007_UUID 15962575136 1027792 15156056752
1% /mnt/lfs[OST:7]</div>
<div><br>
</div>
<div>filesystem_summary: 127700601088 8222108
121248509668 1% /mnt/lfs</div>
</div>
<div><br>
</div>
<div>Structure ias flow:</div>
<div><img src="cid:part1.hqzCXu0J.0u47MVeT@llnl.gov" style="vertical-align: bottom; width: 883px; height: 654px;
border: none;" data-hash="image.png_52934_1" data-aid="1" orgwidth="883" orgheight="654" data-image="1" class="ntes-resize-image"> </div>
<div><br>
</div>
<div>After testing, under the current structure, the write
performance of 500,000 "8k" small files is:</div>
<div>NFSclient1——IOPS：28,000; bandwidth——230MB</div>
<div>NFSclient1——IOPS：27,500; bandwidth——220MB</div>
<div><br>
</div>
<div>Now I want to improve the performance of small files to a
better level，May I ask if there is a better way。</div>
<div><br>
</div>
<div>I have noticed a feature called "MIP-IO" that can improve
small file performance, but I don't know how to deploy this
feature. Is there any way to improve small file performance？</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
lustre-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>
<a class="moz-txt-link-freetext" href="https://urldefense.us/v3/__http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org__;!!G2kpM7uM-TzIFchu!jpxmiee-6Txz4MkvTdXq-z7hM1I20qZ98shFSOwVie2UEZXxjo7aWF6Im3x0pik$">https://urldefense.us/v3/__http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org__;!!G2kpM7uM-TzIFchu!jpxmiee-6Txz4MkvTdXq-z7hM1I20qZ98shFSOwVie2UEZXxjo7aWF6Im3x0pik$</a>
</pre>
</blockquote>
</body>
</html>