[lustre-discuss] Nodemap and setreuid/setregid

[Hans Henrik Happe]

Hi,

Did the test 2.12.4 with the same result. Also, I narrowed it down to
SSK only. It also happens without nodemaps being activated.

@Sebastian: I wonder if you did test this with SSK? I was very focused
on nodemaps being the cause to start with.

Cheers,
Hans Henrik

On 29.02.2020 23.44, Hans Henrik Happe wrote:
> Hi,
> 
> Sorry for the delay. I had to spend some time nursing the glusterfs that
> this lustre fs will replace :-)
> 
> Anyway, I've created a procedure to reproduce the issue. It's attached
> together with the testing program.
> 
> Basically, its a simple single mgs,mdt,oss setup, with a nodemap, that
> maps a client to a fileset. This works fine. However, when turning on
> SSK for cli2mdt the issue appears.
> 
> This was for 2.12.3, I will move on to 2.12.4 just to check.
> 
> Cheers,
> Hans Henrik
> 
> On 06.02.2020 23.08, Hans Henrik Happe wrote:
>> Hi Sebastien,
>>
>> Thanks for looking into this.
>>
>> You are right that nodemap deactivation didn't affect the outcome. I
>> must have made a mistake and cannot reproduce.
>>
>> The uid/gid are on the mds. I can do a sudo to the user and run the test
>> program successfully.
>>
>> I forgot to mention that I use SSK in ski mode.
>>
>> I think I will start from scratch and see if I can reproduce and find
>> out at what point it stops working.
>>
>> Cheers,
>> Hans Henrik
>>
>> On 06.02.2020 18.19, Sebastien Buisson wrote:
>>> Hi,
>>>
>>> I am not able to reproduce your issue. I compiled your C program, in all cases I am not getting Permission Denied.
>>>
>>> You say that it works when you deactivate the nodemap. But given that you have a fileset on your nodemap entry « sif », when you deactivate it you might end up doing IOs in a different directory. So you might compare different things.
>>> Also, does the uid/gid 20501 exist on server side?
>>>
>>> Cheers,
>>> Sebastien.
>>>
>>>> Le 6 févr. 2020 à 14:29, Hans Henrik Happe <happe at nbi.dk> a écrit :
>>>>
>>>> Hi,
>>>>
>>>> Thanks for a very quick reply :-) Here are the map:
>>>>
>>>> # lctl get_param nodemap.sif.*
>>>> nodemap.sif.admin_nodemap=1
>>>> nodemap.sif.audit_mode=1
>>>> nodemap.sif.deny_unknown=0
>>>> nodemap.sif.exports=
>>>> [
>>>>  { nid: 172.25.10.51 at tcp, uuid: 56bb9b04-9bb5-d7b5-3f50-d62804690db1 },
>>>> ]
>>>> nodemap.sif.fileset=/sif
>>>> nodemap.sif.id=2
>>>> nodemap.sif.idmap=
>>>> [
>>>>  { idtype: uid, client_id: 501, fs_id: 20501 },
>>>>  { idtype: gid, client_id: 501, fs_id: 20501 }
>>>> ]
>>>> nodemap.sif.map_mode=both
>>>> nodemap.sif.ranges=
>>>> [
>>>>  { id: 11, start_nid: 172.25.1.28 at tcp, end_nid: 172.25.1.28 at tcp },
>>>>  { id: 10, start_nid: 172.25.1.27 at tcp, end_nid: 172.25.1.27 at tcp },
>>>>  { id: 9, start_nid: 172.25.10.51 at tcp, end_nid: 172.25.10.51 at tcp }
>>>> ]
>>>> nodemap.sif.sepol=
>>>>
>>>> nodemap.sif.squash_gid=20000
>>>> nodemap.sif.squash_uid=20000
>>>> nodemap.sif.trusted_nodemap=0
>>>>
>>>> Cheers,
>>>> Hans Henrik
>>>>
>>>> On 06.02.2020 14.17, Sebastien Buisson wrote:
>>>>> Hi,
>>>>>
>>>>> It might be due to a property on the nodemap you defined.
>>>>> Could you please dump your nodemap definition?
>>>>>
>>>>> Thanks,
>>>>> Sebastien.
>>>>>
>>>>>
>>>>>> Le 6 févr. 2020 à 14:14, Hans Henrik Happe <happe at nbi.dk>
>>>>>>  a écrit :
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Has anyone had success with gocryptfs 1.7.x on top of a Lustre nodemap?
>>>>>>
>>>>>> I've tested with Lustre 2.12.3.
>>>>>>
>>>>>> I found that gocryptfs 1.6 worked. However, with 1.7.x I got a lot of
>>>>>> "Permission denied". I tried all permutations of trusted and admin on
>>>>>> the nodemap.
>>>>>>
>>>>>> By stracing a bit, I've created a small peace of code provoking the issue:
>>>>>>
>>>>>> ---
>>>>>>
>>>>>> #include <unistd.h>
>>>>>> #include <sys/types.h>
>>>>>> #include <fcntl.h>
>>>>>> #include <stdio.h>
>>>>>>
>>>>>> int main() {
>>>>>>  int r;
>>>>>>
>>>>>>  setregid(-1, 501);
>>>>>>  setreuid(-1, 501);
>>>>>>
>>>>>>  r = open("foo", O_CREAT, S_IRWXU);
>>>>>>  if (r < 0) {
>>>>>>    perror("open");
>>>>>>  }
>>>>>>  return 0;
>>>>>> }
>>>>>>
>>>>>> ---
>>>>>>
>>>>>>
>>>>>>
>>>>>> When run as root in a directory owned by uid=501 and gid=501 in a
>>>>>> nodemap based Lustre fs it returns:
>>>>>>
>>>>>> open: Permission denied
>>>>>>
>>>>>> Works when I deactivate nodemap (lctl nodemap_activate 0) or just use a
>>>>>> plain local fs.
>>>>>>
>>>>>> I don't think this is intended behavior for nodemaps, but I might be wrong.
>>>>>>
>>>>>> Cheers,
>>>>>> Hans Henrik
>>>>>> _______________________________________________
>>>>>> lustre-discuss mailing list
>>>>>>
>>>>>> lustre-discuss at lists.lustre.org
>>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> 
> 
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>