<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
In Lustre 2.16.0 there is a configurable parameter to allow a "soft" disabling of setstripe behavior, see LU-17628 "lfs: add lfs_setstripe admin restrict".
<div><br>
</div>
<div>There are also OST Pool Quotas that can be used to limit usage of OSTs on a per-user, per-group, or per-project basis. So if you have e.g. some research groups that can have a directory with the flash OST pool, but others do not, then even if they bypass
the "lfs setstripe" restriction, they would not be able to use capacity in flash OST pool. OST Pools and Pool Quotas have some limitations on how they can be used, but are generally sufficient for most sites.
<div><br>
</div>
<div>Cheers, Andreas<br id="lineBreakAtBeginningOfMessage">
<div><br>
<blockquote type="cite">
<div>On May 12, 2025, at 23:55, Noskov, Dr. Sergey <noskov@uni-mainz.de> wrote:</div>
<br class="Apple-interchange-newline">
<div>
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 14px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Hello Andreas,<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Hello Andreas,<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Thank you for your detailed answer. I think this really brings clarity to our problem.<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Because most of the files are output logs from SLURM jobs etc. that could actually be generated as O_APPEND.<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">A suggestion for further development would be that in Lustre such a mechanism as PFL for administrators, but not changeable in the user space. This would
be very handy for us if we have one metadata structure but different OSTs for different departments.<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Best regards<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;">Sergey Noskov<o:p></o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) currentcolor currentcolor; border-image: none; padding: 3pt 0cm 0cm;">
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span style="font-size: 11pt;"><o:p> </o:p></span></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><b><span style="font-family: Calibri, sans-serif;">Von:<span class="Apple-converted-space"> </span></span></b><span style="font-family: Calibri, sans-serif;">Andreas Dilger <<a href="mailto:adilger@ddn.com" style="color: blue; text-decoration: underline;">adilger@ddn.com</a>><br>
<b>Datum:<span class="Apple-converted-space"> </span></b>Dienstag, 13. Mai 2025 um 05:34<br>
<b>An:<span class="Apple-converted-space"> </span></b>Colin Faber <<a href="mailto:cfaber@gmail.com" style="color: blue; text-decoration: underline;">cfaber@gmail.com</a>><br>
<b>Cc:<span class="Apple-converted-space"> </span></b>Sergey Noskov <<a href="mailto:noskov@uni-mainz.de" style="color: blue; text-decoration: underline;">noskov@uni-mainz.de</a>>, "<a href="mailto:lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a>"
<<a href="mailto:lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a>><br>
<b>Betreff:<span class="Apple-converted-space"> </span></b>Re: [lustre-discuss] Question about PFL<o:p></o:p></span></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">Having the actual file layouts would definitely help. There is one "normal" scenario where the PFL layout will be ignored, which is for files created with "open(O_APPEND)" (typically
log files, or redirected output like "command >> file" ). <span class="Apple-converted-space"> </span><o:p></o:p></div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">For performance and scalability reasons, the append-only files are by default 1-stripe files and do not specify a pool, though this is configurable via "mdd.*.append_stripe_count" and
"mdd.*.append_pool".<o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">Recently this has come up as a point of confusion with observant users, especially with the increasing usage of PFL and multiple pools in the same filesystem. <o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">Two things we are looking to improve in this area (LU-17159):<o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">- mark files that use the "append" layout with a flag so that they can be identified more easily (this would appear in "lfs getstripe output" and the man page)<o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">- use only the first component of the PFL layout for the append files if no explicit append layout is configured. I think this would give more control to users, and be more "intuitive". <o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">Feedback is welcome. <o:p></o:p></div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">Cheers, Andreas<o:p></o:p></div>
</div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><br>
<br>
<o:p></o:p></div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" type="cite">
<p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 12pt; font-family: Aptos, sans-serif;">
On May 12, 2025, at 06:51, Colin Faber via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a>> wrote:<o:p></o:p></p>
</blockquote>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" type="cite">
<div>
<div>You don't mention the version. Also lfs getstripe of the parent directory and one of the files would be helpful as well<o:p></o:p></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><o:p> </o:p></div>
<div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">On Mon, May 12, 2025, 4:18<span style="font-family: Arial, sans-serif;"> </span>AM Noskov, Dr. Sergey via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a>>
wrote:<o:p></o:p></div>
</div>
<blockquote style="border-width: medium medium medium 1pt; border-style: none none none solid; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204); border-image: none; padding: 0cm 0cm 0cm 6pt; margin-left: 4.8pt; margin-right: 0cm;" type="cite">
<div>
<div>
<div>Hello everyone,<o:p></o:p></div>
<div>we have two PFL pools in the lustre.<o:p></o:p></div>
<div>We notice that there are files that are in the directory of pool1, but for some reason they belong to pool2.<o:p></o:p></div>
<div>Would that be possible that PFL does not always work?<o:p></o:p></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"> <o:p></o:p></div>
<div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="">With best regards</span><o:p></o:p></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style=""> </span><o:p></o:p></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style="">Sergey Noskov</span><o:p></o:p></div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"><span lang="EN-US" style=""> </span><o:p></o:p></div>
</div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;"> <o:p></o:p></div>
</div>
</div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" target="_blank" style="color: blue; text-decoration: underline;">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><o:p></o:p></div>
</blockquote>
</div>
<div style="margin: 0cm; font-size: 12pt; font-family: Aptos, sans-serif;">_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline;">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" style="color: blue; text-decoration: underline;">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<div>
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
<div>Cheers, Andreas</div>
<div>—</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud/DDN</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br>
</div>
</div>
</body>
</html>