<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Nov 18, 2017 at 4:20 AM, Stu Midgley <span dir="ltr"><<a href="mailto:sdm900@gmail.com" target="_blank">sdm900@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:monospace,monospace">Thank you both for the documentation. I know how hard it is to maintain. </div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">I've asked that all my admin staff to read it - even if some of it doesn't directly apply to our environment.</div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">What we would like is ell organised, comprehensive, accurate and up to date documenation. Most of the time when I dive into the manual, or other online material, I find it isn't quite right (path's slightly wrong or outdated etc). I also have difficulty finding all the information I want in a single location and in a logical fashon. These aren't new issues and blight all documentation, but having the definitive source in a wiki might open it up to more transparency, greater use and thus, ultimately, being kept up to date, even if its by others outside Intel.</div></div></blockquote><div><br></div><div>Documentation should be treated in the say way as code, i.e. automatically tested. This is not a new idea <a href="https://en.wikipedia.org/wiki/Software_documentation#Literate_programming">https://en.wikipedia.org/wiki/Software_documentation#Literate_programming</a><br></div><div>and with the access to various kinds of virtualization this is feasible now.</div><div>There are Python projects (<a href="https://gitlab.com/ase/ase/tree/master/doc/tutorials">https://gitlab.com/ase/ase/tree/master/doc/tutorials</a>), that make use of this idea thanks to <a href="http://www.sphinx-doc.org">http://www.sphinx-doc.org</a> which allows one to execute embedded Python commands</div><div>during the process of building the documentation in html or pdf formats out of rst (restructured text) files.</div><div>There is a system that stores LFS (Linux from scratch) in an xml format for extraction to be executed <a href="http://www.linuxfromscratch.org/alfs/">http://www.linuxfromscratch.org/alfs/</a> <a href="https://github.com/ojab/jhalfs">https://github.com/ojab/jhalfs</a> but it seems not to be under a continuous automatic testing.</div><div>However, projects like <a href="https://docs.openstack.org/install-guide/">https://docs.openstack.org/install-guide/</a> suprisingly do not use this idea and it takes months to correct a small inconsistency in the documentation <a href="https://bugs.launchpad.net/keystone/+bug/1698455">https://bugs.launchpad.net/keystone/+bug/1698455</a></div><div><br></div><div>It is not very difficult to create a virtual setup consisting of several lustre servers in an unattended way (<a href="https://github.com/marcindulak/vagrant-lustre-tutorial-centos6">https://github.com/marcindulak/vagrant-lustre-tutorial-centos6</a>) and use that</div><div>to test the lustre documentation.</div><div>An alternative to making the lustre documentation executable would be to abstract the basics of lustre using a supported configuration management system (is there any progress about <span style="color:rgb(0,102,33);font-size:14px;white-space:nowrap"><a href="https://www.youtube.com/watch?v=WX00LQLYf2w">https://www.youtube.com/watch?v=WX00LQLYf2w</a> ?) </span>and test that using the standard CI tools.</div><div><br></div><div>Cheers</div><div><br></div><div>Marcin</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">I'd also like a section where people can post their experiences and solutions. For example, in recent times, we have battled bad interactions with ZFS+lustre which lead to poor performance and ZFS corruption. While we have now tuned both lustre and zfs and the bugs have mostly been fixed, the learnings, trouble shooting methods etc. should be preserved and might assist others in the future diagnose tricky problems.</div></div></blockquote><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace">That's my 5c.</div><div style="font-family:monospace,monospace"><br></div><div style="font-family:monospace,monospace"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Nov 18, 2017 at 6:03 AM, Dilger, Andreas <span dir="ltr"><<a href="mailto:andreas.dilger@intel.com" target="_blank">andreas.dilger@intel.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Nov 16, 2017, at 22:41, Cowe, Malcolm J <<a href="mailto:malcolm.j.cowe@intel.com" target="_blank">malcolm.j.cowe@intel.com</a>> wrote:<br>
><br>
> I am pleased to announce the availability of a new systems administration guide for the Lustre file system, which has been published to <a href="http://wiki.lustre.org" rel="noreferrer" target="_blank">wiki.lustre.org</a>. The content can be accessed directly from the front page of the wiki, or from the following URL:<br>
><br>
> <a href="http://wiki.lustre.org/Category:Lustre_Systems_Administration" rel="noreferrer" target="_blank">http://wiki.lustre.org/Categor<wbr>y:Lustre_Systems_Administratio<wbr>n</a><br>
><br>
> The guide is intended to provide comprehensive instructions for the installation and configuration of production-ready Lustre storage clusters. Topics covered:<br>
><br>
> • Introduction to Lustre<br>
> • Lustre File System Components<br>
> • Lustre Software Installation<br>
> • Lustre Networking (LNet)<br>
> • LNet Router Configuration<br>
> • Lustre Object Storage Devices (OSDs)<br>
> • Creating Lustre File System Services<br>
> • Mounting a Lustre File System on Client Nodes<br>
> • Starting and Stopping Lustre Services<br>
> • Lustre High Availability<br>
><br>
> Refer to the front page of the guide for the complete table of contents.<br>
<br>
Malcolm,<br>
thanks so much for your work on this. It is definitely improving the<br>
state of the documentation available today.<br>
<br>
I was wondering if people have an opinion on whether we should remove<br>
some/all of the administration content from the Lustre Operations Manual,<br>
and make that more of a reference manual that contains details of<br>
commands, architecture, features, etc. as a second-level reference from<br>
the wiki admin guide?<br>
<br>
For that matter, should we export the XML Manual into the wiki and<br>
leave it there? We'd have to make sure that the wiki is being indexed<br>
by Google for easier searching before we could do that.<br>
<br>
Cheers, Andreas<br>
<br>
> In addition, for people who are new to Lustre, there is a high-level introduction to Lustre concepts, available as a PDF download:<br>
><br>
> <a href="http://wiki.lustre.org/images/6/64/LustreArchitecture-v4.pdf" rel="noreferrer" target="_blank">http://wiki.lustre.org/images/<wbr>6/64/LustreArchitecture-v4.pdf</a><br>
><br>
><br>
> Malcolm Cowe<br>
> High Performance Data Division<br>
><br>
> Intel Corporation | <a href="http://www.intel.com" rel="noreferrer" target="_blank">www.intel.com</a><br>
><br>
> ______________________________<wbr>_________________<br>
> lustre-discuss mailing list<br>
> <a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
> <a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listin<wbr>fo.cgi/lustre-discuss-lustre.<wbr>org</a><br>
<br>
Cheers, Andreas<br>
--<br>
Andreas Dilger<br>
Lustre Principal Architect<br>
Intel Corporation<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank">lustre-discuss@lists.lustre.or<wbr>g</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/listin<wbr>fo.cgi/lustre-discuss-lustre.<wbr>org</a><span class="gmail-HOEnZb"><font color="#888888"><br>
</font></span></blockquote></div><span class="gmail-HOEnZb"><font color="#888888"><br><br clear="all"><div><br></div>-- <br><div class="gmail-m_2515768125150461237gmail_signature"><div dir="ltr"><div><font face="monospace, monospace">Dr Stuart Midgley<br><a href="mailto:sdm900@gmail.com" target="_blank">sdm900@gmail.com</a></font></div></div></div>
</font></span></div>
<br>______________________________<wbr>_________________<br>
lustre-discuss mailing list<br>
<a href="mailto:lustre-discuss@lists.lustre.org">lustre-discuss@lists.lustre.<wbr>org</a><br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org" rel="noreferrer" target="_blank">http://lists.lustre.org/<wbr>listinfo.cgi/lustre-discuss-<wbr>lustre.org</a><br>
<br></blockquote></div><br></div></div>