[lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)

[Andreas Dilger]

Thomas,
there _are_ potential use cases for for having incomplete layouts now and in the future:
- limiting the size of files (e.g. logs), so that they don't exceed a set limit
- partial HSM restore or FLR mirrors for very large files on fast/local storage

The append problem is something we are aware of and will hopefully have a fix for soon (patch https://review.whamcloud.com/34425 "LU-9341 llite: Do not lock the world for append" if you want to test it and provide some feedback).

Another thought I just had while re-reading LU-9341 is whether it would be better to have the MDS always create files opened with O_APPEND with stripe_count=1?  There is no write parallelism for O_APPEND files, so having multiple stripes doesn't help the writer.  Because the writer also always locks the whole file [0,EOF] then there is no read-write parallelism either, so creating only a single file stripe simplifies things significantly with no real loss.

Minor complications include ensuring the file is created in the right OST pool (e.g. use the pool from the first component of the original layout of a PFL file), what to do if appending to a PFL+DoM file (instantiate a 1-stripe component to EOF?), and what to do for size-limited PFL files (keep current behaviour and the (hopefully 1-2) existing components?

Cheers, Andreas

On Feb 22, 2019, at 10:09, LEIBOVICI Thomas <thomas.leibovici at cea.fr> wrote:
> 
> Hello Patrick,
> 
> Thank you for the quick reply.
> No, I have no particular use-case in mind, I'm just playing around with PFL.
> 
> If this is currently not properly supported, a quick fix could be to prevent the user from creating such incomplete layouts?
> 
> Regards,
> Thomas
> 
> On 2/22/19 5:33 PM, Patrick Farrell wrote:
>> Thomas,
>> 
>> This is expected, but it's also something we'd like to fix - See LU-9341.
>> 
>> Basically, append tries to instantiate the layout from 0 to infinity, and it fails because your layout is incomplete (ie doesn't go to infinity).
>> 
>> May I ask why you're creating a file with an incomplete layout?  Do you have a use case in mind?
>> 
>> - Patrick
>> From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf of LEIBOVICI Thomas <thomas.leibovici at cea.fr>
>> Sent: Friday, February 22, 2019 10:27:48 AM
>> To: lustre-discuss at lists.lustre.org
>> Subject: [lustre-discuss] EINVAL error when writing to a PFL file (lustre 2.12.0)
>>  
>> Hello,
>> 
>> Is it expected to get an error when appending a PFL file made of 2 
>> regions [0 - 1M] and [1M to 6M]
>> even if writing in this range?
>> 
>> I get an error when appending it, even when writting in the very first 
>> bytes:
>> 
>> [root at vm0]# lfs setstripe  -E 1M -c 1 -E 6M -c 2 /mnt/lustre/m_fou3
>> 
>> [root at vm0]# lfs getstripe /mnt/lustre/m_fou3
>> /mnt/lustre/m_fou3
>>    lcm_layout_gen:    2
>>    lcm_mirror_count:  1
>>    lcm_entry_count:   2
>>      lcme_id:             1
>>      lcme_mirror_id:      0
>>      lcme_flags:          init
>>      lcme_extent.e_start: 0
>>      lcme_extent.e_end:   1048576
>>        lmm_stripe_count:  1
>>        lmm_stripe_size:   1048576
>>        lmm_pattern:       raid0
>>        lmm_layout_gen:    0
>>        lmm_stripe_offset: 3
>>        lmm_objects:
>>        - 0: { l_ost_idx: 3, l_fid: [0x100030000:0x9cf:0x0] }
>> 
>>      lcme_id:             2
>>      lcme_mirror_id:      0
>>      lcme_flags:          0
>>      lcme_extent.e_start: 1048576
>>      lcme_extent.e_end:   6291456
>>        lmm_stripe_count:  2
>>        lmm_stripe_size:   1048576
>>        lmm_pattern:       raid0
>>        lmm_layout_gen:    0
>>        lmm_stripe_offset: -1
>> 
>> [root at vm0]# stat -c %s /mnt/lustre/m_fou3
>> 14
>> 
>> * append fails:
>> 
>> [root at vm0]# echo qsdkjqslkdjkj >> /mnt/lustre/m_fou3
>> bash: echo: write error: Invalid argument
>> 
>> # strace indicates that write() gets the error:
>> 
>> write(1, "qsdkjqslkdjkj\n", 14)         = -1 EINVAL (Invalid argument)
>> 
>> * no error in case of an open/truncate:
>> 
>> [root at vm0]# echo qsdkjqslkdjkj > /mnt/lustre/m_fou3
>> 
>> OK
>> 
>> Is it expected or should I open a ticket?
>> 
>> Thomas
>> 
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Cheers, Andreas
---
Andreas Dilger
CTO Whamcloud