[lustre-discuss] Announce: Lustre Systems Administration Guide

DEGREMONT Aurelien aurelien.degremont at cea.fr
Fri Dec 15 08:12:55 PST 2017


My current understanding of Documentation status is that it is hard to 
update and we want to make this easy to encourage people to improve it. 
Correct? I clearly understand this point and it makes sense.

My point of view on this topic, is that MediaWiki is not a good fit for 
an official documentation.

* It is hard to organize pages in MediaWiki. Pages do not have a logical 
flow. Hard to have a sequence of pages or tasks. You just have pages and 
categories. This is quickly a mess.
* Hard to export pages, hard to convert to another format. Documentation 
will be only browsable online. No more external PDF (useful when you do 
not have Internet access on your compute cluster). Plugins exists but 
are hard to install.
* I do not have in mind a good project documentation which is using 
MediaWiki to maintain it.

My understanding of the current workflow for editing doc is:
- create a JIRA ticket
- git clone the source
- edit the docbook doc
- push modification to Gerrit
- wait for approval

My proposal/Features I like with readthedocs-like system are:
- Nice looking HTML documentation (https://docs.readthedocs.io/en/latest/)
- Easy to download in nice looking PDF or HTML version (see links at 
bottom left)
- Could be edited online thanks to "Edit on GitHub" link (top right), 
which will transform your edit into a PR. This is few clicks and online 
editing, in your browser, no need for an explicit git-clone and a terminal.
- Changes could be moderated (thanks to PR)
- Documentation has clear versions (see v1.0.2 link at bottom left 
http://graphite.readthedocs.io/en/1.0.2/ ).
- Documentation is automatically updated when new commits are made, 
thanks to callbacks.

Indeed, this requires a Git repo for documentation management. I agree 
this is similar to what we currently have, but I think we can drop JIRA 
ticket requirement.

Regarding documentation syntax, Sphinx (backend behind readthedocs) 
supports Markdown and ReStructuredText (rst). Both formats are really 
popular at the moment and, even if I used to use docbook in the past and 
was fine with it, I know a lot of people is complaining about its XML 
style syntax. Markdown and RST have a wiki-style syntax which is much 
more user friendly.

It is also possible to deploy our own Sphinx runtime on lustre.org 
servers if we can manage to setup the same feature readthedocs.org put 
on their own.

I hope this helps clarify my point of view.

This does not blame the really good content that was recently put on 
wiki.lustre.org, just the format, not the content :)


Le 15/12/2017 à 09:32, Dilger, Andreas a écrit :
> On Dec 13, 2017, at 09:02, DEGREMONT Aurelien <aurelien.degremont at cea.fr> wrote:
>> Hello
>> My recommendation would be to go for something like Documentation-as-a-code.
>> It is now very easy to deploy an infrastructure to automatically generate nice looking HTML and PDF versions of a rst or markdown documentation thanks to readthedocs.io
> Maybe I'm missing something, but it seems like the Sphynix markup is just a different form of inline text markup compared to DocBook XML that the current manual is using?  Also, we already host the manual repo in Git, and automatically build PDF, HTML, and ePub formats.
> My main thought about moving to a wiki format is that this makes it easier for users to contribute to the manual, since the text could be edited "in place" in a web browser.  Hopefully that would reduce the barrier to entry, and more people would update the manual and improve the places where it is outdated.
>> readthedocs.io uses Sphinx as backend, likewise kernel.org (https://www.kernel.org/doc/html/v4.11/doc-guide/sphinx.html)
>> A github project could be easy plugged to readthedocs, but you could use your own hooks.
> We started with DOxygen markup for the code (I don't know why it wasn't the same as the kernel), but it would probably be possible to write a script to convert the existing code markup to Sphynx if we wanted.
>> If you use GitHub to store your source code, you can easily edit the documentation source code and create a pull request. I know this is not in line with Gerrit usage, but this is an interesting workflow.
> Could you please explain?  It isn't clear to me how this is significantly different from Git+Gerrit?  It is also possible to edit source files directly in Gerrit, though it isn't clear to me if it is possible to _start_ editing a file and submit a patch vs. being able to edit an existing change.  In either case, one still needs to use the inline markup language.  Does Github have a WYSIWYG editor for the manual?
>> I really think we should really uses such technologies which produce very nice looking documentation, which could be easily exports, versionned and coupled with a git repository. See how kernel.org uses it.
> I'm not necessarily against this, but it would be a considerable amount of work to convert the existing manual, and there would have to be a clear benefit.  Do people thing that the Sphynx markup is much easier to use than the DocBook XML? I don't think we use many unusual features in the XML.
> I guess the other important factor would be whether there are people interested to do this work?  I contribute to the existing manual when I get a chance, and don't find the XML very hard to use, but if converting to a new format would convince more people to contribute then this is something we should consider.
> Cheers, Andreas
>> MediaWiki is not a good fit for doc manual like Lustre one. Not easy to browse, not easy to export.
>> My 2 cents
>> Aurélien
>> Le 17/11/2017 à 23:03, Dilger, Andreas a écrit :
>>> On Nov 16, 2017, at 22:41, Cowe, Malcolm J <malcolm.j.cowe at intel.com> wrote:
>>>> I am pleased to announce the availability of a new systems administration guide for the Lustre file system, which has been published to wiki.lustre.org. The content can be accessed directly from the front page of the wiki, or from the following URL:
>>>>   http://wiki.lustre.org/Category:Lustre_Systems_Administration
>>>>   The guide is intended to provide comprehensive instructions for the installation and configuration of production-ready Lustre storage clusters. Topics covered:
>>>>   	• Introduction to Lustre
>>>> 	• Lustre File System Components
>>>> 	• Lustre Software Installation
>>>> 	• Lustre Networking (LNet)
>>>> 	• LNet Router Configuration
>>>> 	• Lustre Object Storage Devices (OSDs)
>>>> 	• Creating Lustre File System Services
>>>> 	• Mounting a Lustre File System on Client Nodes
>>>> 	• Starting and Stopping Lustre Services
>>>> 	• Lustre High Availability
>>>>   Refer to the front page of the guide for the complete table of contents.
>>> Malcolm,
>>> thanks so much for your work on this.  It is definitely improving the
>>> state of the documentation available today.
>>> I was wondering if people have an opinion on whether we should remove
>>> some/all of the administration content from the Lustre Operations Manual,
>>> and make that more of a reference manual that contains details of
>>> commands, architecture, features, etc. as a second-level reference from
>>> the wiki admin guide?
>>> For that matter, should we export the XML Manual into the wiki and
>>> leave it there?  We'd have to make sure that the wiki is being indexed
>>> by Google for easier searching before we could do that.
>>> Cheers, Andreas
>>>> In addition, for people who are new to Lustre, there is a high-level introduction to Lustre concepts, available as a PDF download:
>>>>   http://wiki.lustre.org/images/6/64/LustreArchitecture-v4.pdf
>>>>     Malcolm Cowe
>>>> High Performance Data Division
>>>>   Intel Corporation | www.intel.com
>>>>   _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>> Cheers, Andreas
>>> --
>>> Andreas Dilger
>>> Lustre Principal Architect
>>> Intel Corporation
>>> _______________________________________________
>>> lustre-discuss mailing list
>>> lustre-discuss at lists.lustre.org
>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
> Cheers, Andreas
> --
> Andreas Dilger
> Lustre Principal Architect
> Intel Corporation

More information about the lustre-discuss mailing list