<div dir="ltr">Thank you Jeff. I got the solution for this it is the variation in zfs rather than the lustre because of the parity considered.<div><br></div><div>But the metadata should occupy the 60% for the inode creation which is not happening in the zfs compared with ext4 ldisk.<br><div><br></div><div>Thanks,</div><div>ANS</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jan 1, 2019 at 1:05 PM ANS <<a href="mailto:ans3456@gmail.com">ans3456@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Thank you Jeff. I have created the lustre on ZFS freshly and no other is having access to it. So when mounted it on client it is showing around 40TB variation from the actual space.</div><div><br></div><div>So what could be the reason for this variation of the size.</div><div><br></div><div>Thanks,</div><div>ANS</div><br><div class="gmail_quote"><div dir="ltr">On Tue, Jan 1, 2019 at 12:21 PM Jeff Johnson <<a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto">Very forward versions...especially on ZFS.</div></div><div dir="auto"><br></div><div dir="auto">You build OST volumes in a pool. If no other volumes are defined in a pool then 100% of that pool will be available for the OST volume but the way ZFS works the capacity doesn’t really belong to the OST volume until blocks are allocated for writes. So you have a pool</div><div dir="auto">Of a known size and you’re the admin. As long as nobody else can create a ZFS volume in that pool then all of the capacity in that pool will go to the OST eventually when new writes occur. Keep in mind that the same pool can contain multiple snapshots (if created) so the pool is a “potential capacity” but that capacity could be concurrently allocated to OST volume writes, snapshots and other ZFS volumes (if created)</div><div dir="auto"><br></div><div dir="auto">—Jeff</div><div dir="auto"><br></div><div dir="auto"><br></div><div><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 31, 2018 at 22:20 ANS <<a href="mailto:ans3456@gmail.com" target="_blank">ans3456@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Thanks Jeff. Currently i am using <div><div><br></div><div>modinfo zfs | grep version</div><div>version: 0.8.0-rc2</div><div>rhelversion: 7.4</div></div><div><br></div><div><div>lfs --version</div><div>lfs 2.12.0</div></div><div><br></div><div>And this is a fresh install. So is there any other possibility to show the complete zpool lun has been allocated for lustre alone.</div><div><br></div><div>Thanks,</div><div>ANS</div><div><br></div><div><br></div></div></div></div></div><div dir="ltr"><br><div class="gmail_quote"><div dir="ltr">On Tue, Jan 1, 2019 at 11:44 AM Jeff Johnson <<a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div dir="auto">ANS,</div></div><div dir="auto"><br></div><div dir="auto">Lustre on top of ZFS has to estimate capacities and it’s fairly off when the OSTs are new and empty. As objects are written to OSTs and capacity is consumed it gets the sizing of capacity more accurate. At the beginning it’s so off that it appears to be an error. </div><div dir="auto"><br></div><div dir="auto">What version are you running? Some patches have been added to make this calculation more accurate. </div><div dir="auto"><br></div><div dir="auto">—Jeff</div><div><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 31, 2018 at 22:08 ANS <<a href="mailto:ans3456@gmail.com" target="_blank">ans3456@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr">Dear Team,<div><br></div><div>I am trying to configure lustre with backend ZFS as file system with 2 servers in HA. But after compiling and creating zfs pools</div><div><div><br></div><div>zpool list</div><div>NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT</div><div>lustre-data 54.5T 25.8M 54.5T - 16.0E 0% 0% 1.00x ONLINE -</div><div>lustre-data1 54.5T 25.1M 54.5T - 16.0E 0% 0% 1.00x ONLINE -</div><div>lustre-data2 54.5T 25.8M 54.5T - 16.0E 0% 0% 1.00x ONLINE -</div><div>lustre-data3 54.5T 25.8M 54.5T - 16.0E 0% 0% 1.00x ONLINE -</div><div>lustre-meta 832G 3.50M 832G - 16.0E 0% 0% 1.00x ONLINE -</div><div><br></div><div>and when mounted to client</div><div><br></div><div>lfs df -h<br></div><div><div>UUID bytes Used Available Use% Mounted on</div><div>home-MDT0000_UUID 799.7G 3.2M 799.7G 0% /home[MDT:0]</div><div>home-OST0000_UUID 39.9T 18.0M 39.9T 0% /home[OST:0]</div><div>home-OST0001_UUID 39.9T 18.0M 39.9T 0% /home[OST:1]</div><div>home-OST0002_UUID 39.9T 18.0M 39.9T 0% /home[OST:2]</div><div>home-OST0003_UUID 39.9T 18.0M 39.9T 0% /home[OST:3]</div><div><br></div><div>filesystem_summary: 159.6T 72.0M 159.6T 0% /home</div></div><div><br></div><div>So out of total 54.5TX4=218TB i am getting only 159 TB usable. So can any one give the information regarding this.</div><div><br></div><div>Also from performance prospective what are the zfs and lustre parameters to be tuned.</div></div></div></div></div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div><br></div>-- <br><div dir="ltr" class="gmail-m_-5607974303079504525gmail-m_-1920713786220498905m_-5659918630238410141gmail-m_5348896815748963344gmail-m_9149608888448623632m_2480666676409727911gmail_signature"><div dir="ltr">Thanks,<div>ANS.</div></div></div></div></div></div></div>
_______________________________________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a><br>
</blockquote></div></div>-- <br><div dir="ltr" class="gmail-m_-5607974303079504525gmail-m_-1920713786220498905m_-5659918630238410141gmail-m_5348896815748963344gmail-m_9149608888448623632gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br><a href="https://maps.google.com/?q=4170+Morena+Boulevard,+Suite+C+-+San+Diego,+CA+92117&entry=gmail&source=g" target="_blank">4170 Morena Boulevard, Suite C - San Diego, CA 92117</a><div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_-5607974303079504525gmail-m_-1920713786220498905m_-5659918630238410141gmail-m_5348896815748963344gmail_signature"><div dir="ltr">Thanks,<div>ANS.</div></div></div>
</div>
</blockquote></div></div>-- <br><div dir="ltr" class="gmail-m_-5607974303079504525gmail-m_-1920713786220498905gmail_signature"><div dir="ltr"><div><div dir="ltr">------------------------------<br>Jeff Johnson<br>Co-Founder<br>Aeon Computing<br><br><a href="mailto:jeff.johnson@aeoncomputing.com" target="_blank">jeff.johnson@aeoncomputing.com</a><br><a href="http://www.aeoncomputing.com" target="_blank">www.aeoncomputing.com</a><br>t: 858-412-3810 x1001 f: 858-412-3845<br>m: 619-204-9061<br><br>4170 Morena Boulevard, Suite C - San Diego, CA 92117<div><br></div><div>High-Performance Computing / Lustre Filesystems / Scale-out Storage</div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail-m_-5607974303079504525gmail_signature"><div dir="ltr">Thanks,<div>ANS.</div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Thanks,<div>ANS.</div></div></div>