<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-2" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi <br>
I see you get errno -16<br>
--<br>
<pre>-16 = EBUSY. This say client reconnected to server which already work on
different request from this client. After old rpc from this client will
be finished - client will be reconnected.
</pre>
--<br>
<br>
Did you cleanly stopped lustre? The procedure I use is:<br>
<br>
1) on the client<br>
umount /mnt/<my lustrefs><br>
lustre_rmmod<br>
2) On the OSS<br>
umount /mnt/<my OST><br>
lustre_rmmod<br>
3) On the MDS<br>
umount /mnt/<my MDT><br>
lustre_rmmod<br>
<br>
Make sure that your lustre isn't in the recovery state<br>
Please run on OSS and MDS following commands<br>
cat /proc/fs/lustre/obdfilter/*/recovery_status<br>
cat /proc/fs/lustre/mds/*/recovery_status<br>
<br>
If you see there COMPLETE or INACTIVE it means that Lustre isn't in the
recovery mode.<br>
<br>
Please could you paste here output from command run on the Lustre client<br>
lctl list_nids<br>
and from command run on server:<br>
lctl ping <client nid><br>
<br>
Cheers,<br>
<br>
Wojciech<br>
<br>
<br>
<br>
Lukas Hejtmanek wrote:
<blockquote cite="mid:20090112103139.GE4250@ics.muni.cz" type="cite">
<pre wrap="">Hello,
</pre>
<blockquote type="cite">
<pre wrap="">Hi,
You need to add new Ethernet NID to the Lustre target config logs
Stop your Lustre file system (umount everything)
Run this on all OST(s) and MDT
tunefs.lustre --erase-param --mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp
--writeconf /dev/dev/Scratch_VG/Scratch_1
tunefs.lustre --erase-param --mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp
--writeconf /dev/dev/Scratch_VG/Scratch_2
tunefs.lustre --erase-param --mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp
--writeconf /dev/dev/Scratch_VG/Scratch_3
Above commands erase current lustre configuration logs from the Lustre targets
and write new configuration.
Mount MDT, OSTs and the client and let me know how it works for you.
I also recommend to add modprobe.conf line on the clients, although this is
not necessary in your case, it will make configuration more sane.
options lnet networks=tcp(eth0)
</pre>
</blockquote>
<pre wrap=""><!---->
I did it. Unfortunately, the TCP client still does not work. After remount,
the IB client works.
The TCP client is able to mount and list the file sytem but it is unable to
read, write or create files.
Dmesg shows these errors:
Lustre: 17857:0:(import.c:396:import_select_connection())
spfs-OST0000-osc-ffff8800e2492800: tried all connections, increasing latency
to 26s
Lustre: 17857:0:(import.c:396:import_select_connection()) Skipped 1 previous
similar message
LustreError: 11-0: an error occurred while communicating with 192.168.0.1@tcp.
The ost_connect operation failed with -16
LustreError: Skipped 1 previous similar message
This is how it looks like now:
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_1
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-MDT0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x5
(MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
Permanent disk data:
Target: spfs-MDT0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x5
(MDT MGS )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
exiting before disk write.
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_2
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-OST0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
Permanent disk data:
Target: spfs-OST0000
Index: 0
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
exiting before disk write.
/usr/local/lustre/sbin/tunefs.lustre --print /dev/Scratch_VG/Scratch_3
checking for existing Lustre data: found CONFIGS/mountdata
Reading CONFIGS/mountdata
Read previous values:
Target: spfs-OST0001
Index: 1
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
Permanent disk data:
Target: spfs-OST0001
Index: 1
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x2
(OST )
Persistent mount opts: errors=remount-ro,extents,mballoc
Parameters: <a class="moz-txt-link-abbreviated" href="mailto:mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp">mgsnode=10.0.0.1@o2ib,192.168.0.1@tcp</a>
exiting before disk write.
</pre>
</blockquote>
<br>
</body>
</html>