[lustre-discuss] Nodemap and setreuid/setregid

Sebastien Buisson sbuisson at ddn.com
Tue Mar 3 07:30:28 PST 2020


Hi,

I was focused on nodemaps, so I did not try with SSK.

Cheers,
Sebastien.

> Le 3 mars 2020 à 16:12, Hans Henrik Happe <happe at nbi.dk> a écrit :
> 
> Hi,
> 
> Did the test 2.12.4 with the same result. Also, I narrowed it down to
> SSK only. It also happens without nodemaps being activated.
> 
> @Sebastian: I wonder if you did test this with SSK? I was very focused
> on nodemaps being the cause to start with.
> 
> Cheers,
> Hans Henrik
> 
> On 29.02.2020 23.44, Hans Henrik Happe wrote:
>> Hi,
>> 
>> Sorry for the delay. I had to spend some time nursing the glusterfs that
>> this lustre fs will replace :-)
>> 
>> Anyway, I've created a procedure to reproduce the issue. It's attached
>> together with the testing program.
>> 
>> Basically, its a simple single mgs,mdt,oss setup, with a nodemap, that
>> maps a client to a fileset. This works fine. However, when turning on
>> SSK for cli2mdt the issue appears.
>> 
>> This was for 2.12.3, I will move on to 2.12.4 just to check.
>> 
>> Cheers,
>> Hans Henrik
>> 
>> On 06.02.2020 23.08, Hans Henrik Happe wrote:
>>> Hi Sebastien,
>>> 
>>> Thanks for looking into this.
>>> 
>>> You are right that nodemap deactivation didn't affect the outcome. I
>>> must have made a mistake and cannot reproduce.
>>> 
>>> The uid/gid are on the mds. I can do a sudo to the user and run the test
>>> program successfully.
>>> 
>>> I forgot to mention that I use SSK in ski mode.
>>> 
>>> I think I will start from scratch and see if I can reproduce and find
>>> out at what point it stops working.
>>> 
>>> Cheers,
>>> Hans Henrik
>>> 
>>> On 06.02.2020 18.19, Sebastien Buisson wrote:
>>>> Hi,
>>>> 
>>>> I am not able to reproduce your issue. I compiled your C program, in all cases I am not getting Permission Denied.
>>>> 
>>>> You say that it works when you deactivate the nodemap. But given that you have a fileset on your nodemap entry « sif », when you deactivate it you might end up doing IOs in a different directory. So you might compare different things.
>>>> Also, does the uid/gid 20501 exist on server side?
>>>> 
>>>> Cheers,
>>>> Sebastien.
>>>> 
>>>>> Le 6 févr. 2020 à 14:29, Hans Henrik Happe <happe at nbi.dk> a écrit :
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> Thanks for a very quick reply :-) Here are the map:
>>>>> 
>>>>> # lctl get_param nodemap.sif.*
>>>>> nodemap.sif.admin_nodemap=1
>>>>> nodemap.sif.audit_mode=1
>>>>> nodemap.sif.deny_unknown=0
>>>>> nodemap.sif.exports=
>>>>> [
>>>>> { nid: 172.25.10.51 at tcp, uuid: 56bb9b04-9bb5-d7b5-3f50-d62804690db1 },
>>>>> ]
>>>>> nodemap.sif.fileset=/sif
>>>>> nodemap.sif.id=2
>>>>> nodemap.sif.idmap=
>>>>> [
>>>>> { idtype: uid, client_id: 501, fs_id: 20501 },
>>>>> { idtype: gid, client_id: 501, fs_id: 20501 }
>>>>> ]
>>>>> nodemap.sif.map_mode=both
>>>>> nodemap.sif.ranges=
>>>>> [
>>>>> { id: 11, start_nid: 172.25.1.28 at tcp, end_nid: 172.25.1.28 at tcp },
>>>>> { id: 10, start_nid: 172.25.1.27 at tcp, end_nid: 172.25.1.27 at tcp },
>>>>> { id: 9, start_nid: 172.25.10.51 at tcp, end_nid: 172.25.10.51 at tcp }
>>>>> ]
>>>>> nodemap.sif.sepol=
>>>>> 
>>>>> nodemap.sif.squash_gid=20000
>>>>> nodemap.sif.squash_uid=20000
>>>>> nodemap.sif.trusted_nodemap=0
>>>>> 
>>>>> Cheers,
>>>>> Hans Henrik
>>>>> 
>>>>> On 06.02.2020 14.17, Sebastien Buisson wrote:
>>>>>> Hi,
>>>>>> 
>>>>>> It might be due to a property on the nodemap you defined.
>>>>>> Could you please dump your nodemap definition?
>>>>>> 
>>>>>> Thanks,
>>>>>> Sebastien.
>>>>>> 
>>>>>> 
>>>>>>> Le 6 févr. 2020 à 14:14, Hans Henrik Happe <happe at nbi.dk>
>>>>>>> a écrit :
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> Has anyone had success with gocryptfs 1.7.x on top of a Lustre nodemap?
>>>>>>> 
>>>>>>> I've tested with Lustre 2.12.3.
>>>>>>> 
>>>>>>> I found that gocryptfs 1.6 worked. However, with 1.7.x I got a lot of
>>>>>>> "Permission denied". I tried all permutations of trusted and admin on
>>>>>>> the nodemap.
>>>>>>> 
>>>>>>> By stracing a bit, I've created a small peace of code provoking the issue:
>>>>>>> 
>>>>>>> ---
>>>>>>> 
>>>>>>> #include <unistd.h>
>>>>>>> #include <sys/types.h>
>>>>>>> #include <fcntl.h>
>>>>>>> #include <stdio.h>
>>>>>>> 
>>>>>>> int main() {
>>>>>>> int r;
>>>>>>> 
>>>>>>> setregid(-1, 501);
>>>>>>> setreuid(-1, 501);
>>>>>>> 
>>>>>>> r = open("foo", O_CREAT, S_IRWXU);
>>>>>>> if (r < 0) {
>>>>>>>   perror("open");
>>>>>>> }
>>>>>>> return 0;
>>>>>>> }
>>>>>>> 
>>>>>>> ---
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> When run as root in a directory owned by uid=501 and gid=501 in a
>>>>>>> nodemap based Lustre fs it returns:
>>>>>>> 
>>>>>>> open: Permission denied
>>>>>>> 
>>>>>>> Works when I deactivate nodemap (lctl nodemap_activate 0) or just use a
>>>>>>> plain local fs.
>>>>>>> 
>>>>>>> I don't think this is intended behavior for nodemaps, but I might be wrong.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Hans Henrik
>>>>>>> _______________________________________________
>>>>>>> lustre-discuss mailing list
>>>>>>> 
>>>>>>> lustre-discuss at lists.lustre.org
>>>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>> _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> 
>> 
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org



More information about the lustre-discuss mailing list