[lustre-discuss] shared key security - issue with kernel key possession?

Steve Brasier steveb at stackhpc.com
Thu Feb 27 08:41:05 PST 2020


Thanks Jeremy. I will file a bug - I've confirmed that if I'm actually
ssh'd in as root then `keyctl read` can read the keys created by `mount -t
lustre ... -o skpath=<securedir>`. So that's definitely an issue.

However, using this workaround of actually logging in as root I now get the
following when trying to mount:

lustre-admin lgss_keyring: [11142]:ERROR:lgss_get_service_str(): cannot
resolve hostname from nid 20001c0a8290a

The only reference <https://jira.whamcloud.com/browse/LU-10593>I can find
to this error is IB networks where the interconnect doesn't have an IP.
That's not the issue here and I can `lnetctl ping` the mgs by NID.

Is there some config I'm missing?

many thanks
Steve

http://stackhpc.com/
Please note I work Tuesday to Friday.


On Wed, 26 Feb 2020 at 00:42, Jeremy Filizetti <jeremy.filizetti at gmail.com>
wrote:

> This is a known issue.  I've had this issue pop up after several days for
> no apparent reason.  At this point I'm thinking the keyring will need to be
> configurable to provide a workaround.  Ideally the keys would be added to a
> Lustre specific keyring so everything would be cleaned up when the modules
> are removed  However, the problem there is they need to be loaded before
> the mount system call.
>
> Please file a bug if you have a chance.
>
> Jeremy
>
> On Tue, Feb 25, 2020 at 6:18 AM Steve Brasier <steveb at stackhpc.com> wrote:
>
>> Hi all,
>>
>> I'm trying to configure a lustre 2.12.2 system with SSK and I believe I
>> have a problem with kernel keys which I'd be grateful for any suggestions
>> on.
>>
>> Essentially I have followed the instructions/examples in the docs for SSK
>> except that:
>> - A host "lustre-storage" hosts the MGS, MDT and OST with the fileystem
>> "test_fs1".
>> - A host "lustre-client1" is in a nodeset "lustre_client1".
>> -  cli2ost and cli2mdt rules set as skn
>> (happy to provide more details if required but I think that's the major
>> differences from the examples)
>>
>> Trying to mount the filesystem from the client fails:
>>
>> [centos at lustre-client1 ~]$ sudo mount -t lustre 192.168.41.10 at tcp1:/test_fs1
>> -o skpath=/etc/lustre /mnt/lustre/test_fs1/
>> mount.lustre: mount 192.168.41.10 at tcp1:/test_fs1 at /mnt/lustre/test_fs1
>> failed: Connection refused
>>
>> Looking in /var/log/messages I can see:
>>
>> lustre-client1 lgss_keyring: [18756]:ERROR:sk_create_cred():
>> keyctl_read() failed for key 1073636326: Permission denied
>>
>> And in fact there is a problem reading this key:
>> [centos at lustre-client1 ~]$ sudo keyctl list @u
>> 1 key in keyring:
>> 1073636326: --alswrv     0     0 user: lustre:test_fs1
>> [centos at lustre-client1 ~]$ sudo keyctl read 1073636326
>> keyctl_read_alloc: Permission denied
>>
>> If I try to create a key myself I can see it has the same permissions as
>> 1073636326, and again reading it fails. Some googling led me to this
>> <https://mjg59.dreamwidth.org/37333.html> which suggests there's a
>> fundamental problem using sudo with kernel keys *. I can't be the only
>> person to try to deploy lustre using sudo though surely? So there must be
>> something I'm missing here to make this work.
>>
>> To work around this I tried including "user" in the /etc/fstab options
>> then mounting as a normal user but that fails:
>> [centos at lustre-client1 ~]$ mount /mnt/lustre/test_fs1/
>> mount.lustre: mount 192.168.41.10 at tcp1:/test_fs1 at /mnt/lustre/test_fs1
>> failed: Operation not permitted
>>
>> and in fact it appears lustre doesn't support the user option?
>> Feb 25 11:12:27 lustre-client1 kernel: LustreError: 152-6: Unknown option
>> 'user', won't mount.
>>
>>
>> As I said any help appreciated!
>>
>> Steve
>>
>> * Although that link says key possession is tied to the original user,
>> which would suggest that the key should show up in centos's keyring, which
>> it doesn't.
>>
>> http://stackhpc.com/
>> Please note I work Tuesday to Friday.
>>
>>> _______________________________________________
>> lustre-discuss mailing list
>> lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200227/455e4c63/attachment.html>


More information about the lustre-discuss mailing list