<div dir="ltr">Yao,<div><br></div><div>Glad you have the correct modules working now. Can you explain why you are employing the virtual disk driver with osd-zfs? </div><div><br></div><div>Capacities with Lustre-ZFS are estimated as zfs has functions like compression that can change "capacity". As ZFS volumes are used and filled as Lustre volumes their capacities will be reported more accurately as objects are allocated to storage.</div><div><br></div><div>Running a Lustre command from a client `lfs df -h` will give a better view of Lustre capacities from the client perspective.</div><div><br></div><div>Your use of dev/vda may be adding obscurity and I'm not sure why you would be adding that.</div><div><br></div><div>--Jeff</div><div><br></div><div><br></div><div><br></div><div>--Jeff</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jun 29, 2023 at 10:33 PM Yao Weng <<a href="mailto:wengyao04@gmail.com">wengyao04@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Jeff:<br>Thank you very much ! I install lustre-zfs-dkms and lustre client can mount lustre filesystem. However, the size does not adds-up.<br>I have <br>- msg<br>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>sudo mkfs.lustre --mgs<span> </span>--reformat --backfstype=zfs --fsname=lustre<span> </span>lustre-mgs/mgs /dev/vda2</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>Permanent disk data:</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Target: <span> </span>MGS</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Index:<span> </span>unassigned</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Lustre FS:<span> </span>lustre</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Mount type: zfs</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Flags:<span> </span>0x64</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>(MGS first_time update )</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Persistent mount opts:<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Parameters:</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zpool create -f -O canmount=off lustre-mgs /dev/vda2</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zfs create -o canmount=off<span> </span>lustre-mgs/mgs</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>xattr=sa</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>dnodesize=auto</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Writing lustre-mgs/mgs properties</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:version=1</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:flags=100</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:index=65535</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:fsname=lustre</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:svname=MGS<br><br><br>- mdt<br>
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>sudo mkfs.lustre --mdt --reformat --backfstype=zfs --fsname=lustre --index=0<span> </span>--mgsnode=10.34.0.103@tcp0 lustre-mdt0/mdt0 /dev/vda2</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>Permanent disk data:</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Target: <span> </span>lustre:MDT0000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Index:<span> </span>0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Lustre FS:<span> </span>lustre</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Mount type: zfs</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Flags:<span> </span>0x61</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>(MDT first_time update )</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Persistent mount opts:<span> </span></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Parameters: mgsnode=10.34.0.103@tcp</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zpool create -f -O canmount=off lustre-mdt0 /dev/vda2</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zfs create -o canmount=off<span> </span>lustre-mdt0/mdt0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>xattr=sa</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>dnodesize=auto</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Writing lustre-mdt0/mdt0 properties</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:mgsnode=10.34.0.103@tcp</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:version=1</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:flags=97</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:index=0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:fsname=lustre</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:svname=lustre:MDT0000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">- ost<br>
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">sudo mkfs.lustre --ost --reformat --backfstype=zfs --fsname=lustre --index=0<span> </span>--mgsnode=10.34.0.103@tcp0 lustre-ost0/ost0 /dev/vda2<span> </span></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>Permanent disk data:</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Target: <span> </span>lustre:OST0000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Index:<span> </span>0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Lustre FS:<span> </span>lustre</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Mount type: zfs</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Flags:<span> </span>0x62</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>(OST first_time update )</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Persistent mount opts:<span> </span></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Parameters: mgsnode=10.34.0.103@tcp</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zpool create -f -O canmount=off lustre-ost0 /dev/vda2</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mkfs_cmd = zfs create -o canmount=off<span> </span>lustre-ost0/ost0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>xattr=sa</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>dnodesize=auto</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>recordsize=1M</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Writing lustre-ost0/ost0 properties</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:mgsnode=10.34.0.103@tcp</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:version=1</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:flags=98</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:index=0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:fsname=lustre</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><span> </span>lustre:svname=lustre:OST0000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">I have 51G for /dev/vda</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">df -H /dev/vda2<span> </span></span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem<span> </span>Size<span> </span>Used Avail Use% Mounted on</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)">
</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">devtmpfs <span> </span>51G <span> </span>0 <span> </span>51G <span> </span>0% /dev<br><br>On my client node, <br>
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">sudo mount -t lustre 10.34.0.103@tcp0:/lustre /mnt<br><br>However, the size is only 25M, shouldn't it be 51G ?<br>
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">df -H /mnt</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem <span> </span>Size<span> </span>Used Avail Use% Mounted on</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">10.34.0.103@tcp:/lustre <span> </span>25M<span> </span>3.2M <span> </span>19M<span> </span>15% /mnt<br><br>Thanks<br>Yao</span></p></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 28, 2023 at 12:22 PM Jeff Johnson <<a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Yao,<div><br></div><div>Either add the required kernel-{devel,debuginfo} so the osd-ldiskfs kernel modules can be built against your kernel or remove lustre-all-dkms package and replace with lustre-zfs-dkms and build ZFS only Lustre modules. </div><div><br></div><div>--Jeff</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 28, 2023 at 8:45 AM Yao Weng <<a href="mailto:wengyao04@gmail.com" target="_blank">wengyao04@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I have error when installing <span style="font-variant-ligatures:no-common-ligatures;color:rgb(0,0,0);font-family:Menlo;font-size:11px">lustre-all-dkms-2.15.3-1.el8.noarch<br><br></span><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Loading new lustre-all-2.15.3 DKMS files...</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Deprecated feature: REMAKE_INITRD (/usr/src/lustre-all-2.15.3/dkms.conf)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Building for 4.18.0-477.15.1.el8_8.x86_64 4.18.0-477.10.1.el8_lustre.x86_64</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Building initial module for 4.18.0-477.15.1.el8_8.x86_64</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Deprecated feature: REMAKE_INITRD (/var/lib/dkms/lustre-all/2.15.3/source/dkms.conf)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">realpath: /var/lib/dkms/spl/2.1.12/source: No such file or directory</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">realpath: /var/lib/dkms/spl/kernel-4.18.0-477.15.1.el8_8.x86_64-x86_64: No such file or directory</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">configure: WARNING:</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Disabling ldiskfs support because complete ext4 source does not exist.</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">If you are building using kernel-devel packages and require ldiskfs</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">server support then ensure that the matching kernel-debuginfo-common</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">and kernel-debuginfo-common-<arch> packages are installed.</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">awk: fatal: cannot open file `/var/lib/dkms/lustre-all/2.15.3/build/_lpb/Makefile.compile.lustre' for reading (No such file or directory)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">./configure: line 53751: test: too many arguments</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">./configure: line 53755: test: too many arguments</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Error!<span> </span>Build of osd_ldiskfs.ko failed for: 4.18.0-477.15.1.el8_8.x86_64 (x86_64)</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Make sure the name of the generated module is correct and at the root of the</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">build directory, or consult make.log in the build directory</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">/var/lib/dkms/lustre-all/2.15.3/build for more information.</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">warning: %post(lustre-all-dkms-2.15.3-1.el8.noarch) scriptlet failed, exit status 7</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0);min-height:13px"><span style="font-variant-ligatures:no-common-ligatures"></span><br></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Error in POSTIN scriptlet in rpm package lustre-all-dkms<br><br><br></span></p></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 28, 2023 at 9:57 AM Yao Weng <<a href="mailto:wengyao04@gmail.com" target="_blank">wengyao04@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thank Jeff:<br>My installation steps are<br><br>step1: set local software provision (<a href="https://wiki.lustre.org/Installing_the_Lustre_Software" target="_blank">https://wiki.lustre.org/Installing_the_Lustre_Software</a>)<br>I download all rpm from<br><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server" target="_blank">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server</a></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client" target="_blank">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client</a></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8" style="font-size:13px" target="_blank">https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8</a><br><br>step 2: Install the Lustre e2fsprogs distribution:</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">sudo yum --nogpgcheck --disablerepo=* --enablerepo=e2fsprogs-wc install e2fsprogs</p><p style="margin:0px 0px 2px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>step 3Install EPEL repository support:</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">sudo yum -y install epel-release</p><p style="margin:0px 0px 2px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>step 4 Follow the instructions from the <a href="https://openzfs.github.io/openzfs-docs/Getting%20Started/RHEL-based%20distro/index.html" target="_blank"><span style="color:rgb(220,161,13)">ZFS on Linux project</span></a> to install the ZFS YUM repository definition. Use the DKMS package repository (the default)</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">sudo dnf install <a href="https://zfsonlinux.org/epel/zfs-release-2-3$(rpm" target="_blank">https://zfsonlinux.org/epel/zfs-release-2-3$(rpm</a> --eval "%{dist}").noarch.rpm<br></p><p style="margin:0px 0px 2px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>step 5 Install the Lustre-patched kernel packages. Ensure that the Lustre repository is picked for the kernel packages, by disabling the OS repos:</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>sudo yum --nogpgcheck --disablerepo=base,extras,updates \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">--enablerepo=lustre-server install \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel-devel \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel-headers \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel-tools \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel-tools-libs \</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">kernel-tools-libs-devel</p><p style="margin:0px 0px 2px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>step 6 Generate a persistent hostid on the machine, if one does not already exist. This is needed to help protect ZFS zpools against simultaneous imports on multiple servers. For example:</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">hid=`[ -f /etc/hostid ] && od -An -tx /etc/hostid|sed 's/ //g'`</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">[ "$hid" = `hostid` ] || genhostid</p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">
</p><p style="margin:0px 0px 2px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br>step 7 reboot </p><br>step 8 install lustre and zfs<br><span style="font-family:"Helvetica Neue";font-size:13px">sudo yum --skip-broken --nogpgcheck --enablerepo=lustre-server install \<br></span><span style="font-family:"Helvetica Neue";font-size:13px"> lustre-dkms \<br></span><span style="font-family:"Helvetica Neue";font-size:13px"> lustre-osd-zfs-mount \<br></span><span style="font-family:"Helvetica Neue";font-size:13px"> lustre \<br></span><span style="font-family:"Helvetica Neue";font-size:13px"> lustre-resource-agents \<br> </span><span style="background-color:rgb(248,249,250);color:rgb(0,0,0);font-family:monospace,monospace;font-size:14px">lustre-dkms \</span><span style="font-family:"Helvetica Neue";font-size:13px"><br></span><span style="font-family:"Helvetica Neue";font-size:13px"> zfs<br><br>step 9 </span><span style="color:rgb(32,33,34);font-family:sans-serif;font-size:14px">Load the Lustre and ZFS kernel modules to verify that the software has installed correctly<br></span>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">sudo modprobe -v zfs<br>
</p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:13px;line-height:normal;font-family:"Helvetica Neue"">sudo modprobe -v lustre</p></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jun 28, 2023 at 1:04 AM Jeff Johnson <<a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Did you install the Lustre server RPMs? </div><div dir="auto">Your email lists both server and client repositories. </div><div dir="auto"><br></div><div dir="auto">Are you using DKMS? Did you install and built lustre-zfs-dkms or lustre-all-dkms packages?</div><div dir="auto"><br></div><div dir="auto">It doesn’t appear that you have any Lustre server kernel modules loaded, which makes me suspect you didn’t install or built the server side RPMs or DKMS trees </div><div dir="auto"><br></div><div dir="auto"><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Jun 27, 2023 at 21:41 Yao Weng via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi:<br>I follow <a href="https://wiki.lustre.org/Installing_the_Lustre_Software" target="_blank">https://wiki.lustre.org/Installing_the_Lustre_Software</a> to install lustre.<br><br>My kernel is<br><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">$ uname -r</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">4.18.0-477.13.1.el8_8.x86_64<br><br>I install <br></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server" style="font-family:"Helvetica Neue"" target="_blank">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/server</a></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client" style="font-family:"Helvetica Neue"" target="_blank">https://downloads.whamcloud.com/public/lustre/lustre-2.15.3/el8.8/client</a></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><a href="https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8" style="font-family:"Helvetica Neue"" target="_blank">https://downloads.whamcloud.com/public/e2fsprogs/1.47.0.wc2/el8</a></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""><br></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue"">lsmod | grep lustre<br></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> 1048576 0</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lmv 204800 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">mdc 282624 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lov 344064 2 mdc,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">ptlrpc 2490368 7 fld,osc,fid,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">obdclass 3633152 8 fld,osc,fid,ptlrpc,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lnet 704512 7 osc,obdclass,ptlrpc,ksocklnd,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">lustre</b></span></p><p style="font-size:13px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:"Helvetica Neue""></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;color:rgb(0,0,0)"><span style="font-family:Menlo;font-variant-ligatures:no-common-ligatures">libcfs 266240 11 fld,lnet,osc,fid,obdclass,ptlrpc,ksocklnd,lov,mdc,lmv,</span><span style="font-variant-ligatures:no-common-ligatures;color:rgb(202,51,35)"><b><font face="Menlo" style="font-family:Menlo;color:rgb(202,51,35)">lustre</font><br><br></b><span style="font-family:"Helvetica Neue";font-size:13px;font-variant-ligatures:normal;color:rgb(34,34,34)">lsmod | grep zfs<br></span></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">lsmod | grep zfs</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> 3887104 0</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zunicode 335872 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zzstd 512000 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zlua 176128 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zavl 16384 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">icp 319488 1 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">zcommon 102400 2 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,icp</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">znvpair 90112 2 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,zcommon</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">spl 114688 6 </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(202,51,35)"><b style="font-family:Menlo">zfs</b></span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">,icp,zzstd,znvpair,zcommon,zavl<br><br><br></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal">I am able to create mgs/mdt/ost<br><br>But when I try to mount<br></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">sudo mount.lustre lustre-mgs/mgs /lustre/mnt</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">mount.lustre: mount lustre-mgs/mgs at /lustre/mnt failed: No such device</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">Are the lustre modules loaded?</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"> Check /etc/modprobe.conf and /proc/filesystems<br><br>dmesg gives these error<br></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.604090] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 158-c: Can't load module 'osd-zfs'</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.606174] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(genops.c:361:class_newdev()) OBD: unknown type: osd-zfs</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.607856] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_config.c:620:class_attach()) Cannot create device MGS-osd of type osd-zfs : -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.609805] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_mount.c:195:lustre_start_simple()) MGS-osd attach error -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.611426] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(obd_mount_server.c:1993:server_fill_super()) Unable to start osd on lustre-mgs/mgs: -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo"></span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(57,192,38)">[76783.613457] </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo;color:rgb(170,171,37)">LustreError</span><span style="font-variant-ligatures:no-common-ligatures;font-family:Menlo">: 223535:0:(super25.c:183:lustre_fill_super()) llite: Unable to mount <unknown>: rc = -19</span></p><p style="font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;margin:0px;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(202,51,35)"><br></p></div>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>