[Lustre-discuss] tunefs.lustre --mgsnode non-destructive ?

[Chris Hunter]

We want to reconfigure our lustre for multi-homed network. However the
mgs only appears to respond to clients using the original network (o2ib).

Here is the kernel module (modprobe.conf) line
> options lnet 'networks="o2ib(ib0),tcp(eth0)"'

When mounting lustre on the new (tcp) network, we see the following
syslog messages:
> kernel: LustreError: 10899:0:(events.c:454:ptlrpc_uuid_to_peer()) No 
NID found for 10.2.1.30 at o2ib
> kernel: LustreError: 
10899:0:(client.c:58:ptlrpc_uuid_to_connection()) cannot find peer
10.2.1.30 at o2 ib!
> kernel: LustreError: 10899:0:(ldlm_lib.c:321:client_obd_setup()) 
can't add initial connection
> kernel: LustreError: 
8819:0:(connection.c:144:ptlrpc_put_connection()) NULL connection
> kernel: LustreError: 10899:0:(obd_config.c:332:class_setup()) setup 
lustre0-MDT0000-mdc-ffff81022ea1 cc00 failed (-2)
> kernel: LustreError: 
10899:0:(obd_config.c:1071:class_config_llog_handler()) Err -2 on cfg
command:
> kernel: Lustre:    cmd=cf003 0:lustre0-MDT0000-mdc 
1:lustre0-MDT0000_UUID  2:10.2.1.30 at o2ib
> kernel: LustreError: 15c-8: MGC10.1.1.30 at tcp: The configuration from 
log 'lustre0-client' failed (-2).

After viewing syslogs and searching mailing lists, it appears the mdt
needs to be reformatted to include the new network (tcp) routing to the mgs.

We have a combined mgs+mdt. My best guess, the needed command on the
mds is:
tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt

The original command to make the mdt did not include the mgsnode flag:
tunefs --dryrun -l dev_mdt
> Reading CONFIGS/mountdata
>
>   Read previous values:
> Target:     lustre0-MDT0000
> Index:      0
> Lustre FS:  lustre0
> Mount type: ldiskfs
> Flags:      0x5
>              (MDT MGS )
> Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
> Parameters: mdt.group_upcall=/usr/sbin/l_getgroups

Q. Is tunefs.lustre command this non-destructive ?
tunefs.lustre --mgsnode="IP1 at o2ib0,IP2 at tcp0" dev_mdt

Q. Do I need to update config on the OSTs for our new multi-homed network ?

Thanks,
Chris Hunter