[Lustre-discuss] Singlehomed to multihomed upgrade
Lukas Hejtmanek
xhejtman at ics.muni.cz
Mon Jan 12 02:31:39 PST 2009
Hello,
> Hi,
>
> You need to add new Ethernet NID to the Lustre target config logs
> Stop your Lustre file system (umount everything)
> Run this on all OST(s) and MDT
> tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_1
> tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_2
> tunefs.lustre --erase-param --mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
> --writeconf /dev/dev/Scratch_VG/Scratch_3
>
> Above commands erase current lustre configuration logs from the Lustre targets
> and write new configuration.
>
> Mount MDT, OSTs and the client and let me know how it works for you.
> I also recommend to add modprobe.conf line on the clients, although this is
> not necessary in your case, it will make configuration more sane.
> options lnet networks=tcp(eth0)
I did it. Unfortunately, the TCP client still does not work. After remount,
the IB client works.
The TCP client is able to mount and list the file sytem but it is unable to
read, write or create files.
Dmesg shows these errors:
Lustre: 17857:0:(import.c:396:import_select_connection())
spfs-OST0000-osc-ffff8800e2492800: tried all connections, increasing latency
to 26s
Lustre: 17857:0:(import.c:396:import_select_connection()) Skipped 1 previous
similar message
LustreError: 11-0: an error occurred while communicating with 192.168.0.1 at tcp.
The ost_connect operation failed with -16
LustreError: Skipped 1 previous similar message
This is how it looks like now:
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_1
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-MDT0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x5
(MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
Permanent disk data:
Target: spfs-MDT0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x5
(MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
exiting before disk write.
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_2
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-OST0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
Permanent disk data:
Target: spfs-OST0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
exiting before disk write.
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_3
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-OST0001
Index: 1
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
Permanent disk data:
Target: spfs-OST0001
Index: 1
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: mgsnode=10.0.0.1 at o2ib,192.168.0.1 at tcp
exiting before disk write.
--
Lukáš Hejtmánek
More information about the lustre-discuss
mailing list