[Lustre-discuss] Lustre voltaire configuration

[Ms. Megan Larko]

Hopefully this is a silly question for you.

When you changed your lustre set-up to use the ib0 in place of the
eth1, did you do the updated writeconf command on the MDS/MDT and the
OSS/OST to have them point to the new address?

I'm kind of assuming you have but I thought I would ask the question.

Cheers!
megan

Message: 2
Date: Mon,  5 Oct 2009 14:58:41 +0200 (MEST)
From: Aielli Roberto <r.aielli at cineca.it>
Subject: [Lustre-discuss] Lustre voltaire configuration
To: lustre-discuss at lists.lustre.org
Message-ID: <4AC9ED80.3040001 at cineca.it>
Content-Type: text/plain; charset=ISO-8859-1

Hi,
I'm trying to configure Lustre 1.8.1 with a Voltaire infiniband network.
On the MGS, MDS and OSSs I have two interfaces: eth1 and ib0. I've
successfully  completed a test using eth1 so I've mounted a filesystem
on client node. Now I want to do the same thing with Voltaire infiniband
(ib0) modifying the modprobe.conf on both servers an clients with the line:

options lnet networks=tcp(ib0)


When I try to mount the FS on the client node nothing happen and I find
the following error in the syslog:

Oct  5 13:08:12 xc264 kernel: Lustre:
5468:0:(linux-tcpip.c:688:libcfs_sock_connect()) Error -101 connecting
0.0.0.0/1023 -> 172.31.1.25/988

Oct  5 13:08:12 xc264 kernel: Lustre:
5468:0:(acceptor.c:95:lnet_connect_console_error()) Connection to
172.31.1.25 at tcp at host 172.31.1.25 was unreachable: the network or
that node may be down, or

 Lustre may be misconfigured.

Oct  5 13:08:12 xc264 kernel: Lustre:
5468:0:(socklnd_cb.c:421:ksocknal_txlist_done()) Deleting packet type
1 len 368 172.31.65.24 at tcp->172.31.1.25 at tcp

Oct  5 13:08:17 xc264 kernel: Lustre:
5474:0:(client.c:1383:ptlrpc_expire_one_request()) @@@ Request
x1315690795499541 sent from lustre-MDT0000-mdc-ffff81021f97e400 to NID
172.31.1.25 at tcp 5s ago ha

s timed out (limit 5s).

Oct  5 13:08:17 xc264 kernel:   req at ffff81021828fc00
x1315690795499541/t0 o38->lustre-MDT0000_UUID at 172.31.1.25@tcp:12/10
lens 368/584 e 0 to 1 dl 1254740897 ref 1 fl Rpc:N/0/0 rc 0/0


The main problem is displayed on the first line. The MGS ib0 address is
172.31.65.25 but as you can see the client always try to connect the
eth1 address (172.31.1.25) even shutting down eth1.

Placing in modprobe.conf a line with vib(ib0) the problem is different
I've also tried to modify modprobe.conf by changing the options line to:

options lnet networks=vib(ib0)


but in the syslog I've found:

Oct  5 12:33:19 xc264 kernel: LustreError:
4864:0:(api-ni.c:1043:lnet_startup_lndnis()) Can't load LND vib,
module kviblnd


Are there other configurations for IB that I forgot to make/modify?

Thanks in advance for your help


------------------------------

_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss


End of Lustre-discuss Digest, Vol 45, Issue 5