[Lustre-discuss] Singlehomed to multihomed upgrade

Lukas Hejtmanek xhejtman at ics.muni.cz
Mon Jan 12 02:31:39 PST 2009


Hello,

> Hi,
> 
> You need to add new Ethernet NID to the Lustre target config logs
> Stop your Lustre file system (umount everything)
> Run this on all OST(s) and MDT
>      tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_1
>   tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_2
>  tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_3
>
> Above commands erase current lustre configuration logs from the Lustre targets
> and write new configuration.
>
> Mount MDT, OSTs and the client and let me know how it works for you.
> I also recommend to add modprobe.conf line on the clients, although this is
> not necessary in your case, it will make configuration more sane.
> options lnet networks=tcp(eth0)

I did it. Unfortunately, the TCP client still does not work. After remount,
the IB client works.

The TCP client is able to mount and list the file sytem but it is unable to
read, write or create files.

Dmesg shows these errors:
Lustre: 17857:0:(import.c:396:import_select_connection())
spfs-OST0000-osc-ffff8800e2492800: tried all connections, increasing latency
to 26s
Lustre: 17857:0:(import.c:396:import_select_connection()) Skipped 1 previous
similar message
LustreError: 11-0: an error occurred while communicating with 192.168.0.1 at tcp.
The ost_connect operation failed with -16
LustreError: Skipped 1 previous similar message

This is how it looks like now:
 /usr/local/lustre/sbin/tunefs.lustre  --print /dev/Scratch_VG/Scratch_1
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata

   Read previous values:
Target:     spfs-MDT0000
Index:      0
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x5
              (MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp


   Permanent disk data:
Target:     spfs-MDT0000
Index:      0
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x5
              (MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp

exiting before disk write.

 /usr/local/lustre/sbin/tunefs.lustre  --print /dev/Scratch_VG/Scratch_2
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata

   Read previous values:
Target:     spfs-OST0000
Index:      0
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x2
              (OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp


   Permanent disk data:
Target:     spfs-OST0000
Index:      0
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x2
              (OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp

exiting before disk write.

 /usr/local/lustre/sbin/tunefs.lustre  --print /dev/Scratch_VG/Scratch_3
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata

   Read previous values:
Target:     spfs-OST0001
Index:      1
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x2
              (OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp


   Permanent disk data:
Target:     spfs-OST0001
Index:      1
Lustre FS:  spfs
Mount type: ldiskfs
Flags:      0x2
              (OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp

exiting before disk write.


-- 
Lukáš Hejtmánek



More information about the lustre-discuss mailing list