[Lustre-discuss] Problem with LNET configuration

[Stefano Elmopi]

Hi,

I have a Lustre file system, consisting of a MGS/MDS an two OSS, interconnected with Infiniband.
The version of Lustre is 1.8.3 and the SO of the servers is CentOS 5.4 and I used
the following commands to their formatting:

MGS/MDS:
mkfs.lustre --mgs /dev/mpath/mpath1
mount -t lustre /dev/mpath/mpath1 /MGS
mkfs.lustre --mdt --fsname=lustre01 --mgsnode=172.16.100.111 at tcp0,192.168.150.1 at o2ib0 --mgsnode=172.16.100.121 at tcp0,192.168.150.11 at o2ib0 --failnode=172.16.100.121 at tcp0,192.168.150.11 at o2ib0 /dev/mpath/mpath2
mount -t lustre /dev/mpath/mpath2 /MDS_1/

OSS_1
mkfs.lustre --ost --fsname=lustre01 --failnode=172.16.100.122 at tcp0,192.168.150.12 at o2ib0 --mgsnode=172.16.100.111 at tcp0,192.168.150.1 at o2ib0 --mgsnode=172.16.100.121 at tcp0,192.168.150.11 at o2ib0 /dev/mpath/mpath1
mount -t lustre /dev/mpath/mpath1 /LUSTRE_1

OSS_2
mkfs.lustre --ost --fsname=lustre01 --failnode=172.16.100.121 at tcp0,192.168.150.11 at o2ib0 --mgsnode=172.16.100.111 at tcp0,192.168.150.1 at o2ib0 --mgsnode=172.16.100.121 at tcp0,192.168.150.11 at o2ib0 /dev/mpath/mpath2
mount -t lustre /dev/mpath/mpath2 /LUSTRE_1

and then there are two clients mounted, one on Ethernet and one on IB.
I disconnected the IB cable to simulate the breaking of the IB card on OSS_2.
I modified the file modprobe.conf to start LNET with only Ethernet card and then mount Lustre
filesystem and the operation seems to be successful, the ethernet client can see the entire filesystem.
The problem comes when I try to force a write on OSS_2 because writing crashes ,and the operation goes wrong.
Log on MGS/MDS:

Jul 12 15:04:59 mdt01prdpom kernel: LustreError: 4238:0:(events.c:66:request_out_callback()) @@@ type 4, status -113  req at ffff81013ea52000 x1340531260082684/t0 o8->lustre01-OST0001_UUID at 172.16.100.121@tcp:28/4 lens 368/584 e 0 to 1 dl 1278939908 ref 2 fl Rpc:N/0/0 rc 0/0
Jul 12 15:04:59 mdt01prdpom kernel: LustreError: 4238:0:(events.c:66:request_out_callback()) Skipped 16 previous similar messages
Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4237:0:(lov_request.c:690:lov_update_create_set()) error creating fid 0x10f8004 sub-object on OST idx 1/1: rc = -11
Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4237:0:(lov_request.c:690:lov_update_create_set()) Skipped 1 previous similar message
Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4408:0:(mds_open.c:441:mds_create_objects()) error creating objects for inode 17793028: rc = -5
Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4408:0:(mds_open.c:826:mds_finish_open()) mds_create_objects: rc = -5


My question is:

you can mount the server OSS_2 so that it can provide service with the ethernet card ?
If yes, What should I do?


Thanks



Ing. Stefano Elmopi
Gruppo Darco - Resp. ICT Sistemi
Via Ostiense 131/L Corpo B, 00154 Roma

cell. 3466147165
tel.  0657060500
email:stefano.elmopi at sociale.it

"Ai sensi e per effetti della legge sulla tutela  della  riservatezza personale
(D.lgs n. 196/2003),  questa @mail e' destinata  unicamente alle persone sopra
indicate e le informazioni in essa contenute sono da considerarsi strettamente
riservate. E' proibito leggere, copiare, usare o diffondere il contenuto della
presente @mail  senza  autorizzazione. Se avete ricevuto  questo messaggio per
errore, siete pregati di rispedire la stessa al mittente. Grazie"

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100712/657b70f2/attachment.htm>