[lustre-discuss] error while configuring lnet
Parag Khuraswar
parag_k at citilindia.com
Sun Nov 12 00:16:20 PST 2017
Hi Brett,
I am using MOFED “MLNX_OFED_LINUX-4.1-1.0.2.0” with kernel “3.10.0-514.el7.x86_64”
o/p of modprobe –v ko2iblnd
[root at mds1 ~]# modprobe -v ko2iblnd
install /usr/sbin/ko2iblnd-probe require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193
insmod /lib/modules/3.10.0-514.el7.x86_64/extra/lustre/net/ko2iblnd.ko require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193
modprobe: ERROR: could not insert 'ko2iblnd': Invalid argument
modprobe: ERROR: Error running install command for ko2iblnd
modprobe: ERROR: could not insert 'ko2iblnd': Operation not permitted
[root at mds1 ~]#
Regards,
Parag
From: Brett Lee [mailto:brettlee.lustre at gmail.com]
Sent: Saturday, November , 2017 8:08 PM
To: Parag Khuraswar
Cc: Mannthey, Keith; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] error while configuring lnet
Hi Parag,
You may need to confirm that the in-kernel IB and the IB in the kernel module "match" (are compatible). I think that loading the module (`sudo modprobe -v ko2iblnd`) may be sufficient to verify the match (it's been a while, others may correct me).
Please indicate which kernel and which IB you are using.
Brett
--
Protect yourself against cybercrime
PDS Software Solutions
https://www.TrustPDS.com <https://www.trustpds.com/>
On Fri, Nov 10, 2017 at 8:30 PM, Parag Khuraswar <parag_k at citilindia.com> wrote:
Hi Keith,
Below errors I am getting while adding lnet and mounting mdt.
dmesg logs while adding lnet
=========================================
[317831.432182] LNetError: 28362:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256
=========================================
dmesg logs while mounting mdt
==========================================
[290476.172602] LNetError: 23040:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256
[317478.730515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro
[317480.166277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc
[317480.313296] LustreError: 28268:0:(ldlm_lib.c:483:client_obd_setup()) can't add initial connection
[317480.313600] LustreError: 28268:0:(obd_config.c:608:class_setup()) setup MGC10.2.1.204 at o2ib failed (-2)
[317480.313603] LustreError: 28268:0:(obd_mount.c:202:lustre_start_simple()) MGC10.2.1.204 at o2ib setup error -2
[317480.313632] LustreError: 28268:0:(obd_mount_server.c:1573:server_put_super()) no obd home-MDT0000
[317480.313635] LustreError: 28268:0:(obd_mount_server.c:132:server_deregister_mount()) home-MDT0000 not registered
[317480.433934] Lustre: server umount home-MDT0000 complete
[317480.433940] LustreError: 28268:0:(obd_mount.c:1504:lustre_fill_super()) Unable to mount (-2)
==========================================
Regards,
Parag
From: Mannthey, Keith [mailto:keith.mannthey at intel.com]
Sent: Saturday, November , 2017 2:06 AM
To: Parag Khuraswar; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet
If you have ib0 device check dmesg for more hints on what is going wrong.
Thanks,
Keith
From: Parag Khuraswar [mailto:parag_k at citilindia.com]
Sent: Friday, November 10, 2017 10:59 AM
To: Mannthey, Keith <keith.mannthey at intel.com>; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet
Hi,
Basically I am trying to add lnet. Deleting is just try whether it is happing or not.
Main is I want to add o2ib network. Which is giving error “invalid argument ”
==================
[root at mds2 ~]# lnetctl net add --net o2ib --if ib0
add:
- net:
errno: -22
descr: "cannot add network: Invalid argument"
================
I am really not able to understand what argument is invalid in my command.
I am able to ping ib0 network
Regards,
Parag
From: Mannthey, Keith [mailto:keith.mannthey at intel.com]
Sent: Friday, November , 2017 10:51 PM
To: Parag Khuraswar; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet
What are you trying to accomplish?
>From below:
10.1.1.205 at tcp is on 0 at lo not eno1 and in general you should not need the “—if” option to delete a fabric.
Try: # lnetctl net del --net tcp
Can you do a normal ping over ib0?
“dmesg” can sometime provide greater details about errors like this.
Thanks,
Keith
From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Parag Khuraswar
Sent: Friday, November 10, 2017 9:10 AM
To: lustre-discuss at lists.lustre.org
Subject: [lustre-discuss] error while configuring lnet
Hi,
I am trying to add lnet but getting below error.
======================
[root at mds2 ~]# lnetctl net show
net:
- net type: lo
local NI(s):
- nid: 0 at lo
status: up
- net type: tcp
local NI(s):
- nid: 10.1.1.205 at tcp
status: up
[root at mds2 ~]# lnetctl net add --net o2ib --if ib0
add:
- net:
errno: -22
descr: "cannot add network: Invalid argument"
[root at mds2 ~]# lnetctl net del --net tcp --if eno1
del:
- net:
errno: -22
descr: "cannot del network: Invalid argument"
[root at mds2 ~]# lctl list_nids
10.1.1.205 at tcp
[root at mds2 ~]#
====================================
Regards,
Parag
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171112/e1337cf5/attachment-0001.html>
More information about the lustre-discuss
mailing list