[lustre-discuss] billions of 50k files

[E.S. Rosenberg]

Maybe this would be where multiple MDS + small files on MDS would shine?
My 1 millionth of a bitcoin,
Eli

On Thu, Nov 30, 2017 at 12:31 AM, Brian Andrus <toomuchit at gmail.com> wrote:

> All,
>
> I have always seen lustre as a good solution for large files and not the
> best for many small files.
> Recently, I have seen a request for a small lustre system (2 OSSes, 1 MDS)
> that would be for billions of files that average 50k-100k.
>
> It seems to me, that for this to be 'of worth', the block sizes on disks
> need to be small, but even then, with tcp overhead and inode limitations,
> it may still not perform all that well (compared to larger files).
>
> Am I off here? Have there been some developments in lustre that help this
> scenario (beyond small files being stored on the MDT directly)?
>
> Thanks for any insight,
>
> Brian Andrus
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171130/47e8ab98/attachment.html>