[lustre-discuss] Wrong striping when creating files inside a directory with explicit layout

[Santiago Freire - InCo]

Thank you Andreas for your response.

The application is a rather simple Python script that processes a large 
input file and writes many smaller output files in parallel based on a 
given criteria. I checked the code and it does not create files in a 
temporary directory nor renames them afterward, all output files are 
created directly in a single target directory. I also verified this with 
strace to make sure no unexpected rename operations were taking place.

All the OSTs are up and running. My current setup currently consists of 
one MDT and four OSTs.

The reason I am trying to enforce a specific set of OSTs is that I am 
evaluating the performance benefits of increasing the number of OSTs for 
applications that consume and generate massive amounts of data. Because 
of this I need to experiment with different stripe configurations, using 
1, 2, 3, or 4 OSTs. If you know a better way of achieving this purpose, 
I'd be grateful to hear your suggestions.

Thanks again for your help.

Santiago

On 12/7/25 19:42, Andreas Dilger wrote:
> Two reasons that I know about why this might happen:
> - the application is creating the files in a different directory and 
> then renaming them
>   (for example, MinIO is rename crazy)
> - one of the OSTs is not available at time of creation, and another 
> one is used instead,
>   since stripe_count is more important than specific OST selection
>
> If you want to constrain files to a specific set of OSTs (eg. flash 
> vs. disk) you should create an OST pool.
>
> Might I ask what your specific goal is here? There may be another way 
> to achieve it.
>
> Cheers, Andreas
>
> On Dec 8, 2025, at 03:28, Santiago Freire - InCo via lustre-discuss 
> <lustre-discuss at lists.lustre.org> wrote:
>>
>> Hello everyone.
>>
>> I am observing unexpected behavior regarding file striping in a 
>> directory that has an explicitly configured layout. I'm launching an 
>> application that writes *new *files on a given directory, which is 
>> empty and I previously set its layout using |lfs setstripe| with a 
>> fixed stripe count, stripe size, and a specific set of OSTs (command 
>> lfs setstripe -S 4M -c 3 -o 1,2,3 target_directory). Running simple 
>> tools such as |dd| inside that directory produces files with the 
>> correct stripe pattern, checked using lfs getstripe.
>>
>> However, when my application creates new files in that same 
>> directory, *all* of those files end up with a different layout: some 
>> are created with a stripe count of 1 and almost always put in the OST 
>> with index 1, others are placed n OSTs that are not part of the 
>> directory’s configured offset/OST set (for example, striped across 
>> OSTs 0, 1 and 2). This happens even though the files did not exist 
>> beforehand and the directory was empty with the correct striping 
>> applied. The application creates files with standard POSIX I/O 
>> (|open()| followed by appends and writes), nothing exotic or 
>> MPI-IO-related.
>>
>> Given that the directory layout is correct and tools like |dd| follow 
>> it reliably, I am trying to understand under what circumstances 
>> Lustre would ignore the directory’s default layout when creating new 
>> regular files. I would appreciate any insight or guidance on what 
>> might explain this behaviour, and how could I fix it.
>>
>> I'm using Lustre 2.15.7, with RHEL 9.6 for the clients and RHEL 8.10 
>> for the OSSs/MDS/MGS.
>>
>> Thank you very much in advance.
>>
>> Best regards,
>> Santiago
>>
>>
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20251207/1fb03607/attachment-0001.htm>