[lustre-discuss] Nodemap and setreuid/setregid

Fri May 1 00:03:10 PDT 2020

Hi,

I forgot to send the bug report to the list, so her it is:

https://jira.whamcloud.com/browse/LU-13361

Cheers,
Hans Henrik

On 10.03.2020 09.38, Hans Henrik Happe wrote:
> Hi,
>
> That explains it. I will file a bug report.
>
> Cheers,
> Hans Henrik
>
> On 03.03.2020 16.30, Sebastien Buisson wrote:
>> Hi,
>>
>> I was focused on nodemaps, so I did not try with SSK.
>>
>> Cheers,
>> Sebastien.
>>
>>> Le 3 mars 2020 à 16:12, Hans Henrik Happe <happe at nbi.dk> a écrit :
>>>
>>> Hi,
>>>
>>> Did the test 2.12.4 with the same result. Also, I narrowed it down to
>>> SSK only. It also happens without nodemaps being activated.
>>>
>>> @Sebastian: I wonder if you did test this with SSK? I was very focused
>>> on nodemaps being the cause to start with.
>>>
>>> Cheers,
>>> Hans Henrik
>>>
>>> On 29.02.2020 23.44, Hans Henrik Happe wrote:
>>>> Hi,
>>>>
>>>> Sorry for the delay. I had to spend some time nursing the glusterfs that
>>>> this lustre fs will replace :-)
>>>>
>>>> Anyway, I've created a procedure to reproduce the issue. It's attached
>>>> together with the testing program.
>>>>
>>>> Basically, its a simple single mgs,mdt,oss setup, with a nodemap, that
>>>> maps a client to a fileset. This works fine. However, when turning on
>>>> SSK for cli2mdt the issue appears.
>>>>
>>>> This was for 2.12.3, I will move on to 2.12.4 just to check.
>>>>
>>>> Cheers,
>>>> Hans Henrik
>>>>
>>>> On 06.02.2020 23.08, Hans Henrik Happe wrote:
>>>>> Hi Sebastien,
>>>>>
>>>>> Thanks for looking into this.
>>>>>
>>>>> You are right that nodemap deactivation didn't affect the outcome. I
>>>>> must have made a mistake and cannot reproduce.
>>>>>
>>>>> The uid/gid are on the mds. I can do a sudo to the user and run the test
>>>>> program successfully.
>>>>>
>>>>> I forgot to mention that I use SSK in ski mode.
>>>>>
>>>>> I think I will start from scratch and see if I can reproduce and find
>>>>> out at what point it stops working.
>>>>>
>>>>> Cheers,
>>>>> Hans Henrik
>>>>>
>>>>> On 06.02.2020 18.19, Sebastien Buisson wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I am not able to reproduce your issue. I compiled your C program, in all cases I am not getting Permission Denied.
>>>>>>
>>>>>> You say that it works when you deactivate the nodemap. But given that you have a fileset on your nodemap entry « sif », when you deactivate it you might end up doing IOs in a different directory. So you might compare different things.
>>>>>> Also, does the uid/gid 20501 exist on server side?
>>>>>>
>>>>>> Cheers,
>>>>>> Sebastien.
>>>>>>
>>>>>>> Le 6 févr. 2020 à 14:29, Hans Henrik Happe <happe at nbi.dk> a écrit :
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Thanks for a very quick reply :-) Here are the map:
>>>>>>>
>>>>>>> # lctl get_param nodemap.sif.*
>>>>>>> nodemap.sif.admin_nodemap=1
>>>>>>> nodemap.sif.audit_mode=1
>>>>>>> nodemap.sif.deny_unknown=0
>>>>>>> nodemap.sif.exports=
>>>>>>> [
>>>>>>> { nid: 172.25.10.51 at tcp, uuid: 56bb9b04-9bb5-d7b5-3f50-d62804690db1 },
>>>>>>> ]
>>>>>>> nodemap.sif.fileset=/sif
>>>>>>> nodemap.sif.id=2
>>>>>>> nodemap.sif.idmap=
>>>>>>> [
>>>>>>> { idtype: uid, client_id: 501, fs_id: 20501 },
>>>>>>> { idtype: gid, client_id: 501, fs_id: 20501 }
>>>>>>> ]
>>>>>>> nodemap.sif.map_mode=both
>>>>>>> nodemap.sif.ranges=
>>>>>>> [
>>>>>>> { id: 11, start_nid: 172.25.1.28 at tcp, end_nid: 172.25.1.28 at tcp },
>>>>>>> { id: 10, start_nid: 172.25.1.27 at tcp, end_nid: 172.25.1.27 at tcp },
>>>>>>> { id: 9, start_nid: 172.25.10.51 at tcp, end_nid: 172.25.10.51 at tcp }
>>>>>>> ]
>>>>>>> nodemap.sif.sepol=
>>>>>>>
>>>>>>> nodemap.sif.squash_gid=20000
>>>>>>> nodemap.sif.squash_uid=20000
>>>>>>> nodemap.sif.trusted_nodemap=0
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Hans Henrik
>>>>>>>
>>>>>>> On 06.02.2020 14.17, Sebastien Buisson wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> It might be due to a property on the nodemap you defined.
>>>>>>>> Could you please dump your nodemap definition?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Sebastien.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Le 6 févr. 2020 à 14:14, Hans Henrik Happe <happe at nbi.dk>
>>>>>>>>> a écrit :
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Has anyone had success with gocryptfs 1.7.x on top of a Lustre nodemap?
>>>>>>>>>
>>>>>>>>> I've tested with Lustre 2.12.3.
>>>>>>>>>
>>>>>>>>> I found that gocryptfs 1.6 worked. However, with 1.7.x I got a lot of
>>>>>>>>> "Permission denied". I tried all permutations of trusted and admin on
>>>>>>>>> the nodemap.
>>>>>>>>>
>>>>>>>>> By stracing a bit, I've created a small peace of code provoking the issue:
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>> #include <unistd.h>
>>>>>>>>> #include <sys/types.h>
>>>>>>>>> #include <fcntl.h>
>>>>>>>>> #include <stdio.h>
>>>>>>>>>
>>>>>>>>> int main() {
>>>>>>>>> int r;
>>>>>>>>>
>>>>>>>>> setregid(-1, 501);
>>>>>>>>> setreuid(-1, 501);
>>>>>>>>>
>>>>>>>>> r = open("foo", O_CREAT, S_IRWXU);
>>>>>>>>> if (r < 0) {
>>>>>>>>>   perror("open");
>>>>>>>>> }
>>>>>>>>> return 0;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> ---
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> When run as root in a directory owned by uid=501 and gid=501 in a
>>>>>>>>> nodemap based Lustre fs it returns:
>>>>>>>>>
>>>>>>>>> open: Permission denied
>>>>>>>>>
>>>>>>>>> Works when I deactivate nodemap (lctl nodemap_activate 0) or just use a
>>>>>>>>> plain local fs.
>>>>>>>>>
>>>>>>>>> I don't think this is intended behavior for nodemaps, but I might be wrong.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Hans Henrik
>>>>>>>>> _______________________________________________
>>>>>>>>> lustre-discuss mailing list
>>>>>>>>>
>>>>>>>>> lustre-discuss at lists.lustre.org
>>>>>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>>>> _______________________________________________
>>>>>>> lustre-discuss mailing list
>>>>>>> lustre-discuss at lists.lustre.org
>>>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>> _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200501/f89471c9/attachment.html>