<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
All of the numbers in this example are estimates/approximations to give an idea about the amount of memory that the MDS may need under normal operating circumstances. However, the MDS will also continue to function with more or less memory. The actual amount
of memory in use will change very significantly based on application type, workload, etc. and the numbers "256" and "100,000" are purely examples of how many files might be in use.
<div class=""><br class="">
</div>
<div class="">I'm not sure you can "test" those numbers, because whatever number of files you test with will be the number of files actually in use. You could potentially _measure_ the number of files/locks in use on a large cluster, but again this will be
highly site and application dependent.<br class="">
<div class=""><br class="">
</div>
<div class="">Cheers, Andreas<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 11, 2024, at 01:24, Amin Brick Mover <<a href="mailto:aminbrickmover@gmail.com" class="">aminbrickmover@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">Hi,
Andreas. </span></div>
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class=""><br class="">
</span></div>
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">Thank
you for your reply. </span></div>
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class=""><br class="">
</span></div>
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">Can
I consider 256 files per core as an empirical parameter? And does the parameter '256' need testing based on hardware conditions? Additionally, in the calculation formula "12 interactive clients * 100,000 files * 2KB = 2400 MB," is the number '100,000' files
also an empirical parameter? Do I need to test it. Can I directly use the values '256' and '100,000'?</span><br class="">
</div>
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank" class="">adilger@whamcloud.com</a>> 于2024年3月11日周一 05:47写道：<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="">These numbers are just estimates, you can use values more suitable to your workload.
<div class=""><br class="">
</div>
<div class="">Similarly, 32-core clients may be on the low side these days. NVIDIA DGX nodes have 256 cores, though you may not have 1024 of them.
<div class=""><br class="">
</div>
<div class="">The net answer is that having 64GB+ of RAM is inexpensive these days and improves MDS performance, especially if you compare it to the cost of client nodes that would sit waiting for filesystem access if the MDS is short of RAM. Better to have
too much RAM on the MDS than too little.</div>
<div class=""><br class="">
</div>
<div class="">Cheers, Andreas <br class="">
<div class=""><br class="">
<blockquote type="cite" class="">
<div class="">On Mar 4, 2024, at 00:56, Amin Brick Mover via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" class="">lustre-discuss@lists.lustre.org</a>> wrote:</div>
<br class="">
<div class="">
<div dir="ltr" class="">
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">In
the Lustre Manual 5.5.2.1 section, the examples mentioned:</span></div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px" class="">
<div class="gmail_default"><i class="">For example, for a single MDT on an MDS with 1,024 compute nodes, 12 interactive login nodes, and a</i></div>
<div class="gmail_default"><i class="">20 million file working set (of which 9 million files are cached on the clients at one time):</i></div>
<div class="gmail_default"><i class="">Operating system overhead = 4096 MB (RHEL8)</i></div>
<div class="gmail_default"><i class="">File system journal = 4096 MB</i></div>
<div class="gmail_default"><i class="">1024 * 32-core clients * <font color="#ff0000" class="">
256 files/core</font> * 2KB = 16384 MB</i></div>
<div class="gmail_default"><i class="">12 interactive clients * <font color="#ff0000" class="">
100,000 files</font> * 2KB = 2400 MB</i></div>
<div class="gmail_default"><i class="">20 million file working set * 1.5KB/file = 30720 MB</i></div>
</blockquote>
<div class="gmail_default" style="font-size:large"><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">I'm
curious, how were the two numbers, </span><span style="font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class=""><font color="#ff0000" class="">256
files/core</font></span><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">
and </span><span style="font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class=""><font color="#ff0000" class="">100,000
files</font></span><span style="color:rgb(13,13,13);font-family:Söhne,ui-sans-serif,system-ui,-apple-system,"Segoe UI",Roboto,Ubuntu,Cantarell,"Noto Sans",sans-serif,"Helvetica Neue",Arial,"Apple Color Emoji","Segoe UI Emoji","Segoe UI Symbol","Noto Color Emoji";font-size:16px" class="">,
determined? Why?</span></div>
<br class="">
</div>
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" class="">lustre-discuss@lists.lustre.org</a><br class="">
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank" class="">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br class="">
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div dir="auto" style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; text-decoration: none;" class="">
<div class="">Cheers, Andreas</div>
<div class="">--</div>
<div class=""><span class="gmail_default" style="font-size:large"></span>Andreas Dilger</div>
<div class="">Lustre Principal Architect</div>
<div class="">Whamcloud</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="">
</div>
<br class="">
<br class="">
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</div>
</body>
</html>