<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Are your performance tests on NFS or on native Lustre clients? Native Lustre clients will likely be faster, and with many clients they can create files in parallel, even in the same directory. With a single NFS server they will be limited by the VFS locking
for a single directory.
<div class=""><br class="">
</div>
<div class="">Are you using IB or TCP networking? IB will be faster for low-latency requests.</div>
<div class=""><br class="">
</div>
<div class="">Are you using the Data-on-MDT feature? This can reduce overhead for very small files.</div>
<div class=""><br class="">
</div>
<div class="">Are you using NVMe storage or e.g. SATA SSDs? Based on the OST size it looks like flash of some kind, unless you are using single-HDD OSTs?</div>
<div class=""><br class="">
</div>
<div class="">Cheers, Andreas<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 18, 2023, at 01:44, 王烁斌 via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="line-height: 1.7; font-size: 14px; font-family: Arial;" class="">
<div class="">Hi all,<br class="">
</div>
<div class=""><br class="">
</div>
<div class="">This is my Lustre FS. </div>
<div class="">
<div class="">UUID 1K-blocks Used Available Use% Mounted on</div>
<div class="">ltfs-MDT0000_UUID 307826072 36904 281574768 1% /mnt/lfs[MDT:0]</div>
<div class="">ltfs-MDT0001_UUID 307826072 36452 281575220 1% /mnt/lfs[MDT:1]</div>
<div class="">ltfs-MDT0002_UUID 307826072 36600 281575072 1% /mnt/lfs[MDT:2]</div>
<div class="">ltfs-MDT0003_UUID 307826072 36300 281575372 1% /mnt/lfs[MDT:3]</div>
<div class="">ltfs-OST0000_UUID 15962575136 1027740 15156068868 1% /mnt/lfs[OST:0]</div>
<div class="">ltfs-OST0001_UUID 15962575136 1027780 15156067516 1% /mnt/lfs[OST:1]</div>
<div class="">ltfs-OST0002_UUID 15962575136 1027772 15156074212 1% /mnt/lfs[OST:2]</div>
<div class="">ltfs-OST0003_UUID 15962575136 1027756 15156067860 1% /mnt/lfs[OST:3]</div>
<div class="">ltfs-OST0004_UUID 15962575136 1027728 15156058224 1% /mnt/lfs[OST:4]</div>
<div class="">ltfs-OST0005_UUID 15962575136 1027772 15156057668 1% /mnt/lfs[OST:5]</div>
<div class="">ltfs-OST0006_UUID 15962575136 1027768 15156058568 1% /mnt/lfs[OST:6]</div>
<div class="">ltfs-OST0007_UUID 15962575136 1027792 15156056752 1% /mnt/lfs[OST:7]</div>
<div class=""><br class="">
</div>
<div class="">filesystem_summary: 127700601088 8222108 121248509668 1% /mnt/lfs</div>
</div>
<div class=""><br class="">
</div>
<div class="">Structure ias flow:</div>
<div class=""><span id="cid:ad69d04$1$186f3a691ff$Coremail$w14767780617$163.com"><image.png></span> </div>
<div class=""><br class="">
</div>
<div class="">After testing, under the current structure, the write performance of 500,000 "8k" small files is:</div>
<div class="">NFSclient1——IOPS：28,000; bandwidth——230MB</div>
<div class="">NFSclient1——IOPS：27,500; bandwidth——220MB</div>
<div class=""><br class="">
</div>
<div class="">Now I want to improve the performance of small files to a better level，May I ask if there is a better way。</div>
<div class=""><br class="">
</div>
<div class="">I have noticed a feature called "MIP-IO" that can improve small file performance, but I don't know how to deploy this feature. Is there any way to improve small file performance？</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" class="">lustre-discuss@lists.lustre.org</a><br class="">
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org<br class="">
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>