[Doc-SIG] Python docs in reST

[Ian Bicking]

Martin Blais wrote:
> hmm...  I have come to the following conclusion: there are two types
> of documentation, and they are completely separate beasts, and I feel
> that we should start recognizing this difference.
> 
> 1. reference docs: its structure mirrors the structure of the code,
> and it is often or best automatically generated from the source code
> and comments in the source code.  It does not matter very much if the
> private methods are visible in this documentation;
> 
> 2. user documentation: a manual written with the specific purpose of
> educating a person about how to use the specific module.  its
> structure does not mirror that of the source code at all.  It does not
> describe all the source code, just what is needed.  There are code
> examples.  There might be different manuals for different types of
> users written from different angles.

I think that's a good distinction.  And like several people I'm not very 
interested in 1.

Right now I'm comfortable using stand-alone text files for some kinds of 
documentation, like tutorials.  These often bring together code that 
isn't closely related, cover topics only briefly, and hopefully can be 
doctested to maintain accuracy.

But when it comes to reference documentation I'm struggling.  For Paste 
in particular, automatically generated documentation would be 
unnecessarily intimidating and hard to use, as it would expose all sorts 
of details about the implementation that aren't important from the 
outside.  Short of prefixing 90% of my symbols with _ (which obviously I 
don't want to do ;) there's a lot of stuff something like epydoc 
couldn't recognize as private -- because they aren't really "private" at 
all, just not that "public" either.  (Well, in theory on the global 
level I can use __all__)

As I've been thinking about this, one solution is interfaces.  I'm 
pretty happy with the documentation value of the Wareweb Servlet 
interface, for instance: 
http://svn.pythonpaste.org/Paste/trunk/paste/wareweb/interfaces.py

And I'm happy to maintain that even though it's not a formal interface, 
I'm not using it for anything, and I don't expect any alternate 
implementations.  (Though I'm reluctant to make Paste depend on Zope 
interfaces until they get into core because they include C code; though 
of course the presence of that C code doesn't mean anything to me 
because I still wouldn't *use* those interfaces for anything, so maybe I 
just have to become comfortable with the installation issues.)  Formal 
ways of noting what interfaces are required for certain arguments would 
also be useful.

But that only covers objects.  And while the source of an interface is 
highly readable, converting that to HTML would be useful.

But I have a lot of middling documentation.  It's not doctestable, it's 
not structured so that an interface is useful, the code is in modules I 
don't expect the user to import specifically, and I am wary of just 
creating a separate text file because of API drift; I've found it hard 
to keep track of the information in two places.

My thinking now is that I put these docs in the modules directly -- 
maybe with docstrings with special markings, maybe in a special variable 
(e.g., __manual__) -- and then manually maintain documents that import 
those bits of documentation into a more structured form.  This would 
give me space to include notes between modules (providing continuity and 
notes on the relations between code), and order them the way I prefer. 
If the documentation is able to have intelligent interlinking, that'd 
also be great; my experience with ReST is not great here, as it makes 
strong distinctions between same-page links and external links.  Though 
maybe that could be easily fixed with a two-phase build process; ReST 
identifies the links, outputs HTML, and then I resolve all the links in 
a separate phase.  This mean I wouldn't have to recompile the ReST as 
often too, which is nice since ReST is slow enough that it matters.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org