[lustre-discuss] Does an updated version exist?

[Christopher J. Morrone]

On 08/26/2016 12:48 PM, Oleg Drokin wrote:

> (it's best to be written by third parties anyway since once you work too much
> on some code, you take too many things for granted/think they are obvious,
> and then the end result has gaps that make it hard on the outsiders).

That is an common excuse that I hear around these parts, but I simply do
not accept that axiom.

Yes, there is a kernel of truth that developers can write documentation
that is less than helpful at times for a myriad of reasons.  But none of
those reasons are an adequate excuse for the rather extreme abdication
of documentation responsibility that Lustre tends to adopt.

There are numerous ways to address the issue of the developers taking
too many things for granted, or failing to document because they seem
too obvious.

First of all, I would argue that this is most likely to happen when the
developers do not practice documenting on a regular basis as an
integrated part of the development process.  If it is a common, required
process, many developers will get better at it over time.

Then there are reviews.  Good peer review of documentation will catch
many bad assumptions in the same way that good peer review catches
assumptions and mistakes in code.

And then after the review process, we should understand that design
documentation is a living document that is always growing and improving,
much like the code itself.  When outsiders or new developers point out
unclear parts that remain after reviews, those sections can _still_ be
improved!

The real reason that documentation doesn't happen is not because
developers are incapable of doing it well.  I would argue that a
software developer incapable of documenting their designs, apis, on disk
formats, etc., is a person lacking a significant software development
skill for software projects of any significant size and/or complexity.

I think instead that most of the core Lustre developers don't _want_ to
do it and not enough of the managers of those developers are willing to
_make_ them do it for whatever reasons.

Sure, we _can_ all sit around hoping that some day an external entity
can come along again to cough up a bunch of money to make a one-time
burst at design documents.  That is largely what we're doing, and it
isn't working very well.  That method is pretty clearly not sustainable,
and does not result in maintained documentation.

Will this ever change in Lustre?  Probably not.  We are very set in our
ways.  Change is hard.

But I will happily eat crow if it does.

Chris