[Lustre-discuss] lnet infiniband config

Daniel Kobras kobras at linux.de
Thu Jun 24 14:43:12 PDT 2010


Hi!

On Tue, Jun 22, 2010 at 04:19:08PM +0200, Thomas Roth wrote:
> I'm getting my feet wet in the infiniband lake and of course I run into
> some problems.
> It would seem I got the compilation part of sles11 kernel 2.6.27 +
> Lustre 1.8.3 + ofed 1.4.2 right, because it allows me to see and use the
> infiniband fabric, and because ko2iblnd loads without any complaints.
> 
> In /etc/modprobe.d/lustre (this is a Debian system, hence this subdir of
> modprobe-configs), I have
> > options ip2nets="o2ib0 192.168.0.[1-5]"

If this is a verbatim copy from the config file, then you're lacking the name
of the module, ie. 'options lnet ip2nets=...'. Maybe also double-check with
'modprobe -c' that options get passed on as intended.

> I load lnet and do 'lctl network up', but then 'lctl list_nids' will
> invariably give me only
> > 192.168.0.1 at tcp
> no matter how I twist the modprobe-config (ip2nets="o2ib",
> network="o2ib", network="o2ib(ib0), etc.)
> 
> This is true as long as I have ib0 configured with the IP 192.168.0.1
> Once I unconfigure it, I get, quite expectedly,
> LNET configure error 100: Network is down

So ib0 is the only network interface in the system? In this case, I could
imagine that ksocklnd gets loaded unconditionally, always grabs the first
interface it can get hold of, and just doesn't leave any IB interface for
ko2iblnd when it eventually gets loaded. This is just a shot in the dark, but
you could check by manually loading modules via insmod.

Regards,

Daniel.




More information about the lustre-discuss mailing list