<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><br></div><div><br></div><div><div>Hi,</div><div><br></div><div>I have a Lustre file system, consisting of a MGS/MDS an two OSS, interconnected with Infiniband.</div><div>The version of Lustre is 1.8.3 and the SO of the servers is CentOS 5.4 and I used</div><div>the following commands to their formatting:</div><div><br></div><div>MGS/MDS:</div><div>mkfs.lustre --mgs /dev/mpath/mpath1</div><div>mount -t lustre /dev/mpath/mpath1 /MGS</div><div>mkfs.lustre --mdt --fsname=lustre01 --mgsnode=172.16.100.111@tcp0,192.168.150.1@o2ib0 --mgsnode=172.16.100.121@tcp0,192.168.150.11@o2ib0 --failnode=172.16.100.121@tcp0,192.168.150.11@o2ib0 /dev/mpath/mpath2</div><div>mount -t lustre /dev/mpath/mpath2 /MDS_1/</div><div><br></div><div>OSS_1</div><div>mkfs.lustre --ost --fsname=lustre01 --failnode=172.16.100.122@tcp0,192.168.150.12@o2ib0 --mgsnode=172.16.100.111@tcp0,192.168.150.1@o2ib0 --mgsnode=172.16.100.121@tcp0,192.168.150.11@o2ib0 /dev/mpath/mpath1</div><div>mount -t lustre /dev/mpath/mpath1 /LUSTRE_1</div><div><br></div><div>OSS_2</div><div>mkfs.lustre --ost --fsname=lustre01 --failnode=172.16.100.121@tcp0,192.168.150.11@o2ib0 --mgsnode=172.16.100.111@tcp0,192.168.150.1@o2ib0 --mgsnode=172.16.100.121@tcp0,192.168.150.11@o2ib0 /dev/mpath/mpath2</div><div>mount -t lustre /dev/mpath/mpath2 /LUSTRE_1</div><div><br></div><div>and then there are two clients mounted, one on Ethernet and one on IB.</div><div>I disconnected the IB cable to simulate the breaking of the IB card on OSS_2.</div><div>I modified the file modprobe.conf to start LNET with only Ethernet card and then mount Lustre</div><div>filesystem and the operation seems to be successful, the ethernet client can see the entire filesystem.</div><div>The problem comes when I try to force a write on OSS_2 because writing crashes ,and the operation goes wrong.</div><div>Log on MGS/MDS:</div><div><br></div><div>Jul 12 15:04:59 mdt01prdpom kernel: LustreError: 4238:0:(events.c:66:request_out_callback()) @@@ type 4, status -113 req@ffff81013ea52000 x1340531260082684/t0 o8->lustre01-OST0001_UUID@172.16.100.121@tcp:28/4 lens 368/584 e 0 to 1 dl 1278939908 ref 2 fl Rpc:N/0/0 rc 0/0</div><div>Jul 12 15:04:59 mdt01prdpom kernel: LustreError: 4238:0:(events.c:66:request_out_callback()) Skipped 16 previous similar messages</div><div>Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4237:0:(lov_request.c:690:lov_update_create_set()) error creating fid 0x10f8004 sub-object on OST idx 1/1: rc = -11</div><div>Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4237:0:(lov_request.c:690:lov_update_create_set()) Skipped 1 previous similar message</div><div>Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4408:0:(mds_open.c:441:mds_create_objects()) error creating objects for inode 17793028: rc = -5</div><div>Jul 12 15:06:07 mdt01prdpom kernel: LustreError: 4408:0:(mds_open.c:826:mds_finish_open()) mds_create_objects: rc = -5</div><div><br></div><div><br></div><div>My question is:</div><div><br></div><div>you can mount the server OSS_2 so that it can provide service with the ethernet card ?</div><div>If yes, What should I do?</div><div><br></div><div><br></div><div>Thanks</div></div><div><br></div><div><br></div><br><div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>Ing. Stefano Elmopi</div><div>Gruppo Darco - Resp. ICT Sistemi</div><div>Via Ostiense 131/L Corpo B, 00154 Roma</div><div><br></div><div>cell. 3466147165</div><div>tel. 0657060500</div><div>email:stefano.elmopi@sociale.it</div><div><br></div><div>"Ai sensi e per effetti della legge sulla tutela della riservatezza personale<br>(D.lgs n. 196/2003), questa @mail e' destinata unicamente alle persone sopra<br>indicate e le informazioni in essa contenute sono da considerarsi strettamente<br>riservate. E' proibito leggere, copiare, usare o diffondere il contenuto della<br>presente @mail senza autorizzazione. Se avete ricevuto questo messaggio per<br>errore, siete pregati di rispedire la stessa al mittente. Grazie"</div></div></div></span></div>
</div>
<br></body></html>