[lustre-discuss] How can we push nodemap updates to MDS from MGC ?

[Haarst, Jan van]

Hi All,

We have the following setup:
2 MDS's (active/passive pair), one is the MGC, while the other is the active MDS.
On the MDS that also runs the MGC, we did this:

 lctl nodemap_add_range --name anunna --range 192.168.48.100 at tcp2

This results in the following nodemap ranges:

for node in mds200 mds201;do echo $node;ssh -x $node lctl get_param -R nodemap.anunna.ranges;done
mds200
nodemap.anunna.ranges=
[
 { id: 11, start_nid: 192.168.48.232 at tcp2, end_nid: 192.168.48.232 at tcp2 },
 { id: 10, start_nid: 192.168.48.231 at tcp2, end_nid: 192.168.48.231 at tcp2 },
 { id: 9, start_nid: 192.168.48.249 at tcp2, end_nid: 192.168.48.249 at tcp2 },
 { id: 6, start_nid: 192.168.2.0 at o2ib, end_nid: 192.168.2.255 at o2ib },
 { id: 5, start_nid: 192.168.33.0 at o2ib2, end_nid: 192.168.33.255 at o2ib2 },
 { id: 4, start_nid: 192.168.32.0 at o2ib2, end_nid: 192.168.32.255 at o2ib2 }
]
mds201
nodemap.anunna.ranges=
[
 { id: 12, start_nid: 192.168.48.100 at tcp2, end_nid: 192.168.48.100 at tcp2 },
 { id: 11, start_nid: 192.168.48.232 at tcp2, end_nid: 192.168.48.232 at tcp2 },
 { id: 10, start_nid: 192.168.48.231 at tcp2, end_nid: 192.168.48.231 at tcp2 },
 { id: 9, start_nid: 192.168.48.249 at tcp2, end_nid: 192.168.48.249 at tcp2 },
 { id: 6, start_nid: 192.168.2.0 at o2ib, end_nid: 192.168.2.255 at o2ib },
 { id: 5, start_nid: 192.168.33.0 at o2ib2, end_nid: 192.168.33.255 at o2ib2 },
 { id: 4, start_nid: 192.168.32.0 at o2ib2, end_nid: 192.168.32.255 at o2ib2 }
]

As you can see mds201 (the one running the MGC) nicely has the additional IP range, but the active MDS doesn't.

As one might imagine, this then results in a non working client, we get permission denied errors if we want to access the mount.

As this is our production system, I would like to propagate the new nodemap range to the active MDS from the MGC without having to restart the service, if possible.

I haven't found the command to do this in the documentation, and our friendly LLMs all give very bad advice.

Does anybody here have (a pointer to) a solution?

With kind regards,

--

Jan van Haarst

HPC Administrator

For Anunna/HPC questions, please use https://support.wur.nl (with HPC as service)

Aanwezig: maandag, dinsdag, donderdag & vrijdag

Facilitair Bedrijf, onderdeel van Wageningen University & Research

Afdeling Informatie Technologie

Postbus 59, 6700 AB, Wageningen

Gebouw 116, Akkermaalsbos 12, 6700 WB, Wageningen

http://www.wur.nl/nl/Disclaimer.htm


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20250901/3272616e/attachment.htm>