[lustre-discuss] re-registration of MDTs and OSTs

[Andreas Dilger]

On Oct 18, 2023, at 13:04, Peter Grandi via lustre-discuss <lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>> wrote:

So I have been upgrading my one and only MDT to a larger ZFS
pool, by the classic route of creating a new pool, new MDT, and
then 'zfs send'/zfs receive' for the copy over (BTW for those
who may not be aware 'zfs send' output can be put into a file to
do offline backups of a Lustre ZFS filesystem instance).

At first I just created an empty MGT on the new devices (on a
server with the same NID as the old MGS), with the assumption
that given that MDTs and OSTs have unique (filesystem instance,
target type, index number) triples, with NIDs being just the
address to find the MGS, or where they can be found, they would
just register themselves with the MGT on startup.

But I found that there was a complaint that they were in a
registered state, and the MGT did not have their registration
entries. I am not sure that is the purpose of that check. So
I just copied over the old MGT where they were registered, and
all was fine.

* Is there a way to re-register MDTs and OSTs belonging to a
 given filesystem instance into a new different MGT?

If you run the "writeconf" process documented in the Lustre Manual, the MDT(s)
and OST(s) will re-register themselves with the MGT.

* Is there a purpose to check whether MDTs and OSTs are or not
 registered in a given MGT?

Yes, this prevents the MDTs/OSTs from accidentally becoming part of a
different filesystem that might have been incorrectly formatted with the
same fsname (e.g. "lustre" has been used as the fsname more than
once).

* Is there a downside to register MDTs and OSTs in a different
 MGT from that which they were registered with initially?

Not too much.  The new MGT will not have any of the old configuration
parameters, but running "writeconf" will also reset any "conf_param"
parameters so not much different (but it will not reset "set_param -P"
parameters).

My guess is that the MGT does not just contain the identities
and addresses of MDTs and OSTs of one or more filesystem
instance, but also a parameter list

If so, is there are way to dump the parameter for a filesystem
instance so it can be restored to a different MGT?

Yes, the "lctl --device MGS llog_print CONFIG_LOG" command
will dump all of the config commands for a particular MDT/OST
or the "params" log for "set_param -P".

The parameters can be restored from a file with "lctl set_param -F".

See the lctl-set_param.8 and lctl-llog_print.8 man pages for details.

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20231024/2cc7801d/attachment.htm>