[lustre-discuss] SSK key refresh issue

[Jeremy Filizetti]

There is nothing I can think of off the top of my head that would be a
limiting factor preventing SSK from working with multiple file systems.
Unfortunately my test environment is unavailable to look at this right
now.  If you can send some server logs I can take a look although I suspect
Sebastien is more familiar with this code than I am now.

Jeremy

On Wed, Jun 3, 2020 at 8:01 PM Hans Henrik Happe <happe at nbi.dk> wrote:

> We only have one MGS with two filesystems. It evolved into to two due to
> some reconfiguration and the end game is to remove one. Anyway, one of the
> modifications to the new one was to have daily key refresh. Either that
> made the refresh issue more likely or having two filesystems is not a good
> idea. It seemed to work with two so we went on and started to rsync some
> data over. Then we hit the refresh issue. For now I'm just asking if
> multiple filesystems could cause issues.
>
> Anyway, I'm setting up a debug system to test if I can reproduce it with a
> single fs. Then I'll get back with more info about the actual error.
>
> I've attached some client output from the failing system. This was on a
> nodemap with a 120s expire key, for fast testing. It seems that there need
> to be I/O during the refresh to hit this.
>
> A bit more info about the setup. All except mgs is configured for ski
> messaging. mdt2ost is using the default nodemap.
>
> Cheers,
> Hans Henrik
>
>
> On 03.06.2020 18.01, Sebastien Buisson wrote:
>
> Hi,
>
> Do you use one shared MGS for all your file systems, or does each file system have its own MGS? In the latter case, are the MGSes running on the same node?
>
> You are mentioning a key refresh issue, so I am wondering if you see this issue with multiple file systems only, or if it occurs when you have just one file system setup?
>
> Cheers,
> Sebastien.
>
>
> Le 3 juin 2020 à 15:07, Hans Henrik Happe <happe at nbi.dk> <happe at nbi.dk> a écrit :
>
> Hi,
>
> I'm trying to hunt down an issue where SSK is failing key refresh on
> 2.12.4. Mounting the filesystem works, but active sessions dies at refresh.
>
> First I would like to get a few things cleared.
>
> Is multiple Lustre filesystems on the same servers supported with SSK?
>
> If so, is it supported to use the same nodemap on each filesystem?
> Obviously, with different keys for each fs.
>
> A mount from an ssh to the root account will create this keyring on
> CentOS 7:
>
> # keyctl show
> Session Keyring
> 669565440 --alswrv      0     0  keyring: _ses
> 458158660 --alswrv      0 65534   \_ keyring: _uid.0
> 129939379 --alswrv      0     0       \_ user: lustre:erda
>
> 65534 usually is nfsnobody but is does not exist on the system. Would
> this be an issue? Even if nfsnobody existed?
>
> A mount through sudo will create this keyring on CentOS 7:
>
> # keyctl show
> Session Keyring
> 381836048 --alswrv      0 65534  keyring: _uid_ses.0
> 423400032 --alswrv      0 65534   \_ keyring: _uid.0
> 934942793 --alswrv      0     0       \_ user: lustre:erda
>
> Again is this a problem?
>
>
> Cheers,
> Hans Henrik
> _______________________________________________
> lustre-discuss mailing listlustre-discuss at lists.lustre.orghttp://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20200603/940d469a/attachment.html>