[lustre-discuss] [EXTERNAL] mdt and mgt are always in RO mode

[Mohr, Rick]

Ran,

I don't know much about multi-tenancy setups in Lustre, but I might be able to point you in the direction of some info.

Lustre doesn't have a way to create users on the client side.  It relies on the uids sent to the server by the client.  In basic setups, the lustre servers will use the same set of usernames/uids as the clients (maybe using LDAP or something similar).  There is an identity upcall that can be set on the server side to help with resolving group memberships if the default behavior isn't working.  See https://doc.lustre.org/lustre_manual.xhtml#identity_upcall for more info on that.

If the uids on the clients don't match those on the server, then you can use the nodemap feature to remap them.  Lustre nodemaps are also used to control access to portions of the file system.  There was a presentation at LUG this year that might help you (https://srcc.stanford.edu/sites/g/files/sbiybj25536/files/media/file/lug25-lustre_multitenancy-buisson_v1.2.pdf).  The subdirs that you were asked to create are not subdirs on the mdt itself.  Those will need to be created within the lustre filesystem from a client.  If a client mounts the filesystem on /lustre, then you will need to make the dirs /lustre/tenantA and /lustre/tenantB on the client (this requires a client that can access the whole filesystem).  Isolating access will come from using lustre's subdir mount feature to mount only a portion of the filesystem namespace (like /lustre/tenantA) on a client.  Nodemap will be used to limit access to those different subdir namespaces.

You might want to look at the lustre manual section on configuring Shared Secret Keys (https://doc.lustre.org/lustre_manual.xhtml#lustressk) and possibly Kerberos (https://doc.lustre.org/lustre_manual.xhtml#managingSecurity.kerberos) too if needed.  Hopefully that is enough to get you started.

--Rick


On 8/6/25, 12:43 AM, "Ran Mo" <HenryMo2012 at outlook.com <mailto:HenryMo2012 at outlook.com>> wrote:

Thanks Rick a lot. You save me from keeping reinstalling Lustre/troubleshooting why it is in RO. :)

Sorry, I am new to Lustre. I am trying to setup and test two functions below. 
1) Use Lustre's identity feature to create users on the client side, and require user to be authenticated at Lustre server n before allowing them to access the file system.
2) Enable the nodemap feature to enforce user data isolation; Create Tenant A and Tenant B, and ensure that each tenant can only mount and see data within their respective subdirectories of the file system.


For test 1), per research , looks like there is a need to install Lustre_id at client side but I just cannot find it.
For test 2), per research , ChatGPT asked me to create two sub directories (for examples, mkdir /mnt/mdt/{tenantA,tenantB}) under mount point of MDT (/mnt/mdt) on MDS server but since it is RO mode, I am not allowed to do so. That's where I have been stuck for test 2). (:

Not sure if you can share your ideas on these.

Environment:
3 Rocky8.10 VMs (MDS+MGS on server1, the other two are OSS). Two Ubuntu VMs (22.04) as client. All is running Lustre 2.15.7.




Thanks again. 
Ran