<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
For many small output files, you are always best off to use 1-stripe files.
<div><br>
</div>
<div>These days, the best way to handle this is by using a default PFL layout which has the first component larger than the "small" file size (eg. 1GB) and then stripes the "large" input file across more OSTs, like:</div>
<div><br>
</div>
<div> lfs setstripe -E 1G -c 1 -E 16G -c 4 -E eof -c 40 DIR<br>
<div><br>
</div>
<div>If you want to ensure that one or more OSTs is not being used at all, you can set "osp.FSNAME-OSTxxxx.create_count=0" on the MDS. </div>
<div><br id="lineBreakAtBeginningOfSignature">
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
<blockquote type="cite">On Dec 8, 2025, at 10:08, Santiago Freire - InCo <sfreire@fing.edu.uy> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<p>Thank you Andreas for your response.</p>
<p>The application is a rather simple Python script that processes a large input file and writes many smaller output files in parallel based on a given criteria. I checked the code and it does not create files in a temporary directory nor renames them afterward,
all output files are created directly in a single target directory. I also verified this with strace to make sure no unexpected rename operations were taking place.</p>
<p>All the OSTs are up and running. My current setup currently consists of one MDT and four OSTs. </p>
<p>The reason I am trying to enforce a specific set of OSTs is that I am evaluating the performance benefits of increasing the number of OSTs for applications that consume and generate massive amounts of data. Because of this I need to experiment with different
stripe configurations, using 1, 2, 3, or 4 OSTs. If you know a better way of achieving this purpose, I'd be grateful to hear your suggestions.</p>
<p>Thanks again for your help.</p>
<p>Santiago</p>
<div class="moz-cite-prefix">On 12/7/25 19:42, Andreas Dilger wrote:<br>
</div>
<blockquote type="cite" cite="mid:A3FAE468-3290-4A68-A64A-8F3354664029@thelustrecollective.com">
Two reasons that I know about why this might happen:
<div>- the application is creating the files in a different directory and then renaming them</div>
<div> (for example, MinIO is rename crazy)</div>
<div>- one of the OSTs is not available at time of creation, and another one is used instead,</div>
<div> since stripe_count is more important than specific OST selection</div>
<div><br>
</div>
<div>If you want to constrain files to a specific set of OSTs (eg. flash vs. disk) you should create an OST pool.</div>
<div><br>
</div>
<div>Might I ask what your specific goal is here? There may be another way to achieve it. </div>
<div><br id="lineBreakAtBeginningOfSignature">
<div dir="ltr">Cheers, Andreas</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">On Dec 8, 2025, at 03:28, Santiago Freire - InCo via lustre-discuss
<a class="moz-txt-link-rfc2396E" href="mailto:lustre-discuss@lists.lustre.org"><lustre-discuss@lists.lustre.org></a> wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<p data-start="197" data-end="212">Hello everyone.</p>
<p data-start="214" data-end="630">I am observing unexpected behavior regarding file striping in a directory that has an explicitly configured layout. I'm launching an application that writes
<b>new </b>files on a given directory, which is empty and I previously set its layout using
<code data-start="420" data-end="435">lfs setstripe</code> with a fixed stripe count, stripe size, and a specific set of OSTs (command
<font face="monospace">lfs setstripe -S 4M -c 3 -o 1,2,3 target_directory</font>). Running simple tools such as
<code data-start="533" data-end="537">dd</code> inside that directory produces files with the correct stripe pattern, checked using
<font face="monospace">lfs getstripe</font>.</p>
<p>However, when my application creates new files in that same directory, <b>all</b> of those files end up with a different layout: some are created with a stripe count of 1 and almost always put in the OST with index 1, others are placed n OSTs that are not
part of the directory’s configured offset/OST set (for example, striped across OSTs 0, 1 and 2). This happens even though the files did not exist beforehand and the directory was empty with the correct striping applied. The application creates files with standard
POSIX I/O (<code data-start="1032" data-end="1040">open()</code> followed by appends and writes), nothing exotic or MPI-IO-related.</p>
<p>Given that the directory layout is correct and tools like <code data-start="1167" data-end="1171">
dd</code> follow it reliably, I am trying to understand under what circumstances Lustre would ignore the directory’s default layout when creating new regular files. I would appreciate any insight or guidance on what might explain this behaviour, and how could
I fix it.</p>
<p>I'm using Lustre 2.15.7, with RHEL 9.6 for the clients and RHEL 8.10 for the OSSs/MDS/MGS.</p>
<p data-start="1948" data-end="1979">Thank you very much in advance.</p>
<p data-start="1981" data-end="2005">Best regards,<br data-start="1994" data-end="1997">
Santiago</p>
<p><br>
</p>
<span>_______________________________________________</span><br>
<span>lustre-discuss mailing list</span><br>
<span><a class="moz-txt-link-abbreviated" href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a></span><br>
<span><a class="moz-txt-link-freetext" href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a></span><br>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</body>
</html>