[lustre-discuss] About Lustre small files performace(8k) improve

[Andreas Dilger]

Are your performance tests on NFS or on native Lustre clients?  Native Lustre clients will likely be faster, and with many clients they can create files in parallel, even in the same directory.  With a single NFS server they will be limited by the VFS locking for a single directory.

Are you using IB or TCP networking?  IB will be faster for low-latency requests.

Are you using the Data-on-MDT feature?  This can reduce overhead for very small files.

Are you using NVMe storage or e.g. SATA SSDs?  Based on the OST size it looks like flash of some kind, unless you are using single-HDD OSTs?

Cheers, Andreas

On Mar 18, 2023, at 01:44, 王烁斌 via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>> wrote:

Hi all,

This is my Lustre FS.
UUID                   1K-blocks        Used   Available Use% Mounted on
ltfs-MDT0000_UUID      307826072       36904   281574768   1% /mnt/lfs[MDT:0]
ltfs-MDT0001_UUID      307826072       36452   281575220   1% /mnt/lfs[MDT:1]
ltfs-MDT0002_UUID      307826072       36600   281575072   1% /mnt/lfs[MDT:2]
ltfs-MDT0003_UUID      307826072       36300   281575372   1% /mnt/lfs[MDT:3]
ltfs-OST0000_UUID    15962575136     1027740 15156068868   1% /mnt/lfs[OST:0]
ltfs-OST0001_UUID    15962575136     1027780 15156067516   1% /mnt/lfs[OST:1]
ltfs-OST0002_UUID    15962575136     1027772 15156074212   1% /mnt/lfs[OST:2]
ltfs-OST0003_UUID    15962575136     1027756 15156067860   1% /mnt/lfs[OST:3]
ltfs-OST0004_UUID    15962575136     1027728 15156058224   1% /mnt/lfs[OST:4]
ltfs-OST0005_UUID    15962575136     1027772 15156057668   1% /mnt/lfs[OST:5]
ltfs-OST0006_UUID    15962575136     1027768 15156058568   1% /mnt/lfs[OST:6]
ltfs-OST0007_UUID    15962575136     1027792 15156056752   1% /mnt/lfs[OST:7]

filesystem_summary:  127700601088     8222108 121248509668   1% /mnt/lfs

Structure ias flow:
<image.png>

After testing, under the current structure, the write performance of 500,000 "8k" small files is:
NFSclient1——IOPS：28,000;  bandwidth——230MB
NFSclient1——IOPS：27,500;  bandwidth——220MB

Now I want to improve the performance of small files to a better level，May I ask if there is a better way。

I have noticed a feature called "MIP-IO" that can improve small file performance, but I don't know how to deploy this feature. Is there any way to improve small file performance？



_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20230328/0c2cabd2/attachment-0001.htm>