[lustre-discuss] Announce: Lustre Systems Administration Guide

Michael Di Domenico mdidomenico4 at gmail.com
Fri Dec 15 08:54:22 PST 2017 

i don't really have a card in this documentation game.  i would,
however, like to address one point that a lot of people over look.
While wiki's are wonderful for documentation, the main downfall is
that you have to have internet access to get to them.  some of us work
in area's where internet access is not available
consistently/conveniently or AT ALL.  having any documentation
included with the source tree (even out of date documentation) is
better then no documentation...

all to often, i'm coming across open source software where the only
included documentation is a readme file that says go look at the
wiki...



On Fri, Dec 15, 2017 at 11:12 AM, DEGREMONT Aurelien
<aurelien.degremont at cea.fr> wrote:
> Hi
>
> My current understanding of Documentation status is that it is hard to
> update and we want to make this easy to encourage people to improve it.
> Correct? I clearly understand this point and it makes sense.
>
> My point of view on this topic, is that MediaWiki is not a good fit for an
> official documentation.
>
> * It is hard to organize pages in MediaWiki. Pages do not have a logical
> flow. Hard to have a sequence of pages or tasks. You just have pages and
> categories. This is quickly a mess.
> * Hard to export pages, hard to convert to another format. Documentation
> will be only browsable online. No more external PDF (useful when you do not
> have Internet access on your compute cluster). Plugins exists but are hard
> to install.
> * I do not have in mind a good project documentation which is using
> MediaWiki to maintain it.
>
> My understanding of the current workflow for editing doc is:
> - create a JIRA ticket
> - git clone the source
> - edit the docbook doc
> - push modification to Gerrit
> - wait for approval
>
> My proposal/Features I like with readthedocs-like system are:
> - Nice looking HTML documentation (https://docs.readthedocs.io/en/latest/)
> - Easy to download in nice looking PDF or HTML version (see links at bottom
> left)
> - Could be edited online thanks to "Edit on GitHub" link (top right), which
> will transform your edit into a PR. This is few clicks and online editing,
> in your browser, no need for an explicit git-clone and a terminal.
> - Changes could be moderated (thanks to PR)
> - Documentation has clear versions (see v1.0.2 link at bottom left
> http://graphite.readthedocs.io/en/1.0.2/ ).
> - Documentation is automatically updated when new commits are made, thanks
> to callbacks.
>
> Indeed, this requires a Git repo for documentation management. I agree this
> is similar to what we currently have, but I think we can drop JIRA ticket
> requirement.
>
> Regarding documentation syntax, Sphinx (backend behind readthedocs) supports
> Markdown and ReStructuredText (rst). Both formats are really popular at the
> moment and, even if I used to use docbook in the past and was fine with it,
> I know a lot of people is complaining about its XML style syntax. Markdown
> and RST have a wiki-style syntax which is much more user friendly.
>
> It is also possible to deploy our own Sphinx runtime on lustre.org servers
> if we can manage to setup the same feature readthedocs.org put on their own.
>
> I hope this helps clarify my point of view.
>
>
> This does not blame the really good content that was recently put on
> wiki.lustre.org, just the format, not the content :)
>
>
> Aurélien
>
>
> Le 15/12/2017 à 09:32, Dilger, Andreas a écrit :
>>
>> On Dec 13, 2017, at 09:02, DEGREMONT Aurelien <aurelien.degremont at cea.fr>
>> wrote:
>>>
>>> Hello
>>>
>>> My recommendation would be to go for something like
>>> Documentation-as-a-code.
>>> It is now very easy to deploy an infrastructure to automatically generate
>>> nice looking HTML and PDF versions of a rst or markdown documentation thanks
>>> to readthedocs.io
>>
>> Maybe I'm missing something, but it seems like the Sphynix markup is just
>> a different form of inline text markup compared to DocBook XML that the
>> current manual is using?  Also, we already host the manual repo in Git, and
>> automatically build PDF, HTML, and ePub formats.
>>
>> My main thought about moving to a wiki format is that this makes it easier
>> for users to contribute to the manual, since the text could be edited "in
>> place" in a web browser.  Hopefully that would reduce the barrier to entry,
>> and more people would update the manual and improve the places where it is
>> outdated.
>>
>>> readthedocs.io uses Sphinx as backend, likewise kernel.org
>>> (https://www.kernel.org/doc/html/v4.11/doc-guide/sphinx.html)
>>>
>>> A github project could be easy plugged to readthedocs, but you could use
>>> your own hooks.
>>
>> We started with DOxygen markup for the code (I don't know why it wasn't
>> the same as the kernel), but it would probably be possible to write a script
>> to convert the existing code markup to Sphynx if we wanted.
>>
>>> If you use GitHub to store your source code, you can easily edit the
>>> documentation source code and create a pull request. I know this is not in
>>> line with Gerrit usage, but this is an interesting workflow.
>>
>> Could you please explain?  It isn't clear to me how this is significantly
>> different from Git+Gerrit?  It is also possible to edit source files
>> directly in Gerrit, though it isn't clear to me if it is possible to _start_
>> editing a file and submit a patch vs. being able to edit an existing change.
>> In either case, one still needs to use the inline markup language.  Does
>> Github have a WYSIWYG editor for the manual?
>>
>>> I really think we should really uses such technologies which produce very
>>> nice looking documentation, which could be easily exports, versionned and
>>> coupled with a git repository. See how kernel.org uses it.
>>
>> I'm not necessarily against this, but it would be a considerable amount of
>> work to convert the existing manual, and there would have to be a clear
>> benefit.  Do people thing that the Sphynx markup is much easier to use than
>> the DocBook XML? I don't think we use many unusual features in the XML.
>>
>> I guess the other important factor would be whether there are people
>> interested to do this work?  I contribute to the existing manual when I get
>> a chance, and don't find the XML very hard to use, but if converting to a
>> new format would convince more people to contribute then this is something
>> we should consider.
>>
>> Cheers, Andreas
>>
>>> MediaWiki is not a good fit for doc manual like Lustre one. Not easy to
>>> browse, not easy to export.
>>>
>>>
>>> My 2 cents
>>>
>>> Aurélien
>>>
>>> Le 17/11/2017 à 23:03, Dilger, Andreas a écrit :
>>>>
>>>> On Nov 16, 2017, at 22:41, Cowe, Malcolm J <malcolm.j.cowe at intel.com>
>>>> wrote:
>>>>>
>>>>> I am pleased to announce the availability of a new systems
>>>>> administration guide for the Lustre file system, which has been published to
>>>>> wiki.lustre.org. The content can be accessed directly from the front page of
>>>>> the wiki, or from the following URL:
>>>>>   http://wiki.lustre.org/Category:Lustre_Systems_Administration
>>>>>   The guide is intended to provide comprehensive instructions for the
>>>>> installation and configuration of production-ready Lustre storage clusters.
>>>>> Topics covered:
>>>>>         • Introduction to Lustre
>>>>>         • Lustre File System Components
>>>>>         • Lustre Software Installation
>>>>>         • Lustre Networking (LNet)
>>>>>         • LNet Router Configuration
>>>>>         • Lustre Object Storage Devices (OSDs)
>>>>>         • Creating Lustre File System Services
>>>>>         • Mounting a Lustre File System on Client Nodes
>>>>>         • Starting and Stopping Lustre Services
>>>>>         • Lustre High Availability
>>>>>   Refer to the front page of the guide for the complete table of
>>>>> contents.
>>>>
>>>> Malcolm,
>>>> thanks so much for your work on this.  It is definitely improving the
>>>> state of the documentation available today.
>>>>
>>>> I was wondering if people have an opinion on whether we should remove
>>>> some/all of the administration content from the Lustre Operations
>>>> Manual,
>>>> and make that more of a reference manual that contains details of
>>>> commands, architecture, features, etc. as a second-level reference from
>>>> the wiki admin guide?
>>>>
>>>> For that matter, should we export the XML Manual into the wiki and
>>>> leave it there?  We'd have to make sure that the wiki is being indexed
>>>> by Google for easier searching before we could do that.
>>>>
>>>> Cheers, Andreas
>>>>
>>>>> In addition, for people who are new to Lustre, there is a high-level
>>>>> introduction to Lustre concepts, available as a PDF download:
>>>>>   http://wiki.lustre.org/images/6/64/LustreArchitecture-v4.pdf
>>>>>     Malcolm Cowe
>>>>> High Performance Data Division
>>>>>   Intel Corporation | www.intel.com
>>>>>   _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>
>>>> Cheers, Andreas
>>>> --
>>>> Andreas Dilger
>>>> Lustre Principal Architect
>>>> Intel Corporation
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>
>> Cheers, Andreas
>> --
>> Andreas Dilger
>> Lustre Principal Architect
>> Intel Corporation
>>
>>
>>
>>
>>
>>
>>
>
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

More information about the lustre-discuss mailing list