<div dir="auto">Did you install the Lustre server RPMs? </div><div dir="auto">Your email lists both server and client repositories. </div><div dir="auto"><br></div><div dir="auto">Are you using DKMS? Did you install and built lustre-zfs-dkms or lustre-all-dkms packages?</div><div dir="auto"><br></div><div dir="auto">It doesn’t appear that you have any Lustre server kernel modules loaded, which makes me suspect you didn’t install or built the server side RPMs or DKMS trees </div><div dir="auto"><br></div><div dir="auto"><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)"><div dir="ltr">Hi:<br>I follow <a href="https://wiki.lustre.org/Installing_the_Lustre_Software" target="_blank">https://wiki.lustre.org/Installing_the_Lustre_Software</a> to install lustre.<br><br>My kernel is<br><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">$ uname -r</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">4.18.0-477.13.1.el8_8.x86_64<br><br>I install <br></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server" target="_blank" style="font-family:"Helvetica Neue"">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server</a></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client" target="_blank" style="font-family:"Helvetica Neue"">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client</a></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8" target="_blank" style="font-family:"Helvetica Neue"">https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8</a></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">lsmod | grep lustre<br></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> 1048576 0</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lmv 204800 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">mdc 282624 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lov 344064 2 mdc,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">ptlrpc 2490368 7 fld,osc,fid,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">obdclass 3633152 8 fld,osc,fid,ptlrpc,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lnet 704512 7 osc,obdclass,ptlrpc,ksocklnd,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;color:rgb(0,0,0)"><span style="font-family:Menlo;font-variant-ligatures:no-common-ligatures">libcfs 266240 11 fld,lnet,osc,fid,obdclass,ptlrpc,ksocklnd,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b><font face="Menlo" style="font-family:Menlo;color:rgb(202,51,35)">lustre</font><br><br></b><span style="font-family:"Helvetica Neue";font-size:13px;font-variant-ligatures:normal;color:rgb(34,34,34)">lsmod | grep zfs<br></span></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lsmod | grep zfs</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> 3887104 0</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zunicode 335872 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zzstd 512000 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zlua 176128 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zavl 16384 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">icp 319488 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zcommon 102400 2 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,icp</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">znvpair 90112 2 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,zcommon</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">spl 114688 6 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,icp,zzstd,znvpair,zcommon,zavl<br><br><br></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal">I am able to create mgs/mdt/ost<br><br>But when I try to mount<br></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">sudo mount.lustre lustre-mgs/mgs /lustre/mnt</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">mount.lustre: mount lustre-mgs/mgs at /lustre/mnt failed: No such device</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">Are the lustre modules loaded?</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> Check /etc/modprobe.conf and /proc/filesystems<br><br>dmesg gives these error<br></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.604090] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 158-c: Can't load module 'osd-zfs'</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.606174] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(genops.c:361:class_newdev()) OBD: unknown type: osd-zfs</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.607856] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_config.c:620:class_attach()) Cannot create device MGS-osd of type osd-zfs : -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.609805] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_mount.c:195:lustre_start_simple()) MGS-osd attach error -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.611426] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_mount_server.c:1993:server_fill_super()) Unable to start osd on lustre-mgs/mgs: -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.613457] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(super25.c:183:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><br></p></div>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>