[lustre-discuss] billions of 50k files

E.S. Rosenberg esr+lustre at mail.hebrew.edu
Wed Nov 29 15:11:32 PST 2017

Maybe this would be where multiple MDS + small files on MDS would shine?
My 1 millionth of a bitcoin,

On Thu, Nov 30, 2017 at 12:31 AM, Brian Andrus <toomuchit at gmail.com> wrote:

> All,
> I have always seen lustre as a good solution for large files and not the
> best for many small files.
> Recently, I have seen a request for a small lustre system (2 OSSes, 1 MDS)
> that would be for billions of files that average 50k-100k.
> It seems to me, that for this to be 'of worth', the block sizes on disks
> need to be small, but even then, with tcp overhead and inode limitations,
> it may still not perform all that well (compared to larger files).
> Am I off here? Have there been some developments in lustre that help this
> scenario (beyond small files being stored on the MDT directly)?
> Thanks for any insight,
> Brian Andrus
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171130/47e8ab98/attachment.html>

More information about the lustre-discuss mailing list