[lustre-discuss] error while configuring lnet

Parag Khuraswar parag_k at citilindia.com
Sun Nov 12 00:16:20 PST 2017


Hi Brett,

 

I am using MOFED “MLNX_OFED_LINUX-4.1-1.0.2.0” with kernel “3.10.0-514.el7.x86_64”

 

o/p of modprobe –v ko2iblnd

 

[root at mds1 ~]# modprobe -v ko2iblnd

install /usr/sbin/ko2iblnd-probe require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193

insmod /lib/modules/3.10.0-514.el7.x86_64/extra/lustre/net/ko2iblnd.ko require_privileged_port=0 use_privileged_port=0 timeout=150 retry_count=7 map_on_demand=32 peer_credits=63 concurrent_sends=63 ntx=32768 credits=32768 fmr_pool_size=8193

modprobe: ERROR: could not insert 'ko2iblnd': Invalid argument

modprobe: ERROR: Error running install command for ko2iblnd

modprobe: ERROR: could not insert 'ko2iblnd': Operation not permitted

[root at mds1 ~]#

 

 

Regards,

Parag

 

 

From: Brett Lee [mailto:brettlee.lustre at gmail.com] 
Sent: Saturday, November , 2017 8:08 PM
To: Parag Khuraswar
Cc: Mannthey, Keith; lustre-discuss at lists.lustre.org
Subject: Re: [lustre-discuss] error while configuring lnet

 

Hi Parag,

 

You may need to confirm that the in-kernel IB and the IB in the kernel module "match" (are compatible).  I think that loading the module (`sudo modprobe -v ko2iblnd`) may be sufficient to verify the match (it's been a while, others may correct me).

 

Please indicate which kernel and which IB you are using.




Brett

--

Protect yourself against cybercrime

PDS Software Solutions

https://www.TrustPDS.com <https://www.trustpds.com/> 

 

On Fri, Nov 10, 2017 at 8:30 PM, Parag Khuraswar <parag_k at citilindia.com> wrote:

Hi Keith,

 

Below errors I am getting while adding lnet and mounting mdt.

 

dmesg logs while adding lnet 

=========================================

[317831.432182] LNetError: 28362:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256

=========================================

 

 

 

dmesg logs while mounting mdt

==========================================

[290476.172602] LNetError: 23040:0:(api-ni.c:1861:lnet_startup_lndnet()) Can't load LND o2ib, module ko2iblnd, rc=256

[317478.730515] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: errors=remount-ro

[317480.166277] LDISKFS-fs (dm-1): mounted filesystem with ordered data mode. Opts: user_xattr,errors=remount-ro,no_mbcache,nodelalloc

[317480.313296] LustreError: 28268:0:(ldlm_lib.c:483:client_obd_setup()) can't add initial connection

[317480.313600] LustreError: 28268:0:(obd_config.c:608:class_setup()) setup MGC10.2.1.204 at o2ib failed (-2)

[317480.313603] LustreError: 28268:0:(obd_mount.c:202:lustre_start_simple()) MGC10.2.1.204 at o2ib setup error -2

[317480.313632] LustreError: 28268:0:(obd_mount_server.c:1573:server_put_super()) no obd home-MDT0000

[317480.313635] LustreError: 28268:0:(obd_mount_server.c:132:server_deregister_mount()) home-MDT0000 not registered

[317480.433934] Lustre: server umount home-MDT0000 complete

[317480.433940] LustreError: 28268:0:(obd_mount.c:1504:lustre_fill_super()) Unable to mount  (-2)

==========================================

 

 

Regards,

Parag

 

 

From: Mannthey, Keith [mailto:keith.mannthey at intel.com] 
Sent: Saturday, November , 2017 2:06 AM


To: Parag Khuraswar; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet

 

If you have ib0 device check dmesg for more hints on what is going wrong. 

 

Thanks,

Keith 

From: Parag Khuraswar [mailto:parag_k at citilindia.com] 
Sent: Friday, November 10, 2017 10:59 AM
To: Mannthey, Keith <keith.mannthey at intel.com>; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet

 

Hi,

 

Basically I am trying to add lnet. Deleting is just try whether it is happing or not.

Main is I want to add o2ib network. Which is giving error “invalid argument ”

==================

[root at mds2 ~]# lnetctl net add --net o2ib --if ib0

add:

    - net:

          errno: -22

          descr: "cannot add network: Invalid argument"

================

I am really not able to understand what argument is invalid in my command.

I am able to ping ib0 network

 

Regards,

Parag

 

 

From: Mannthey, Keith [mailto:keith.mannthey at intel.com] 
Sent: Friday, November , 2017 10:51 PM
To: Parag Khuraswar; lustre-discuss at lists.lustre.org
Subject: RE: [lustre-discuss] error while configuring lnet

 

What are you trying to accomplish? 

 

>From below:

 

10.1.1.205 at tcp is on 0 at lo not eno1 and in general you should not need the “—if” option to delete a fabric. 

 

Try: # lnetctl net del --net tcp

 

Can you do a normal ping over ib0? 

 

“dmesg” can sometime provide greater details about errors like this. 

 

Thanks,

Keith 

 

 

From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Parag Khuraswar
Sent: Friday, November 10, 2017 9:10 AM
To: lustre-discuss at lists.lustre.org
Subject: [lustre-discuss] error while configuring lnet

 

Hi,

 

I am trying to add lnet but getting below error. 

======================

[root at mds2 ~]# lnetctl net show

net:

    - net type: lo

      local NI(s):

        - nid: 0 at lo

          status: up

    - net type: tcp

      local NI(s):

        - nid: 10.1.1.205 at tcp

          status: up

[root at mds2 ~]# lnetctl net add --net o2ib --if ib0

add:

    - net:

          errno: -22

          descr: "cannot add network: Invalid argument"

[root at mds2 ~]# lnetctl net del --net tcp --if eno1

del:

    - net:

          errno: -22

          descr: "cannot del network: Invalid argument"

[root at mds2 ~]# lctl list_nids

10.1.1.205 at tcp

[root at mds2 ~]#

====================================

 

 

Regards,

Parag

 

 


_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20171112/e1337cf5/attachment-0001.html>


More information about the lustre-discuss mailing list