[Lustre-discuss] o2ib module prevents shutdown

[Michael Sternberg]

Hello,

Not sure if this is the right forum:  I'm encountering difficulties  
with o2ib which prevents an LNET shutdown from proceeding:

	Unloading OpenIB kernel modules:NET: Unregistered protocal family 27
	Failed to unload rdma_cm
	Failed to unload rdma_cm
	Failed to unload ib_cm
	Failed to unload ib_sa
	LustreError: 131-3: Received notification of device removal
	Please shutdown LNET to allow this to proceed

This happens on server and client nodes alike.  We run RHEL5.1 and  
OFED 1.2, kernel 2.6.18-53.1.13.el5_lustre.1.6.4.3smp from CFS/Sun.

I narrowed it down to module ko2iblnd, which I attempt to remove  
first (added to PRE_UNLOAD_MODULES in /etc/init.d/openibd), but it  
doesn't work.  Strangely, in "lsmod" the use count of the module is  
one, but I don't see where it's used.


# umount /mnt/lustre
# ifconfig ib0 down
# modprobe -r ko2iblnd
FATAL: Module ko2iblnd is in use.
# lsmod | grep ko2
ko2iblnd              143136  1
lnet                  258088  5 lustre,ksocklnd,ko2iblnd,ptlrpc,obdclass
libcfs                189784  12  
osc,mgc,lustre,lov,lquota,mdc,ksocklnd,ko2iblnd,ptlrpc,obdclass,lnet,lvf 
s
rdma_cm                65940  4 ko2iblnd,ib_iser,rdma_ucm,ib_sdp
ib_core                88576  16  
ko2iblnd,ib_iser,rdma_ucm,ib_ucm,ib_srp,ib_sdp,rdma_cm,ib_cm,iw_cm,ib_lo 
cal_sa,ib_ipoib,ib_sa,ib_uverbs,ib_umad,ib_mthca,ib_mad


I'd be grateful for any hints.



Regards, Michael