[AstroPy] Documentation Guidelines

Mon Jul 11 19:27:24 EDT 2011

Just a quick note regarding the numpy sphinx ext. It can be installed
as a standalone package from

http://pypi.python.org/pypi/numpydoc/0.3.1

(though I'm not sure how up to date that is). Maybe it can also be
directly imported from numpy? (I haven't delved deep enough into this)

I've created a fake package and docs here to demonstrate the various
formats we are talking about:

https://github.com/astrofrog/astropy_doc_test

Note, this is just a personal repo, nothing official. You need to
clone the repo and open doc/build/index.html. Note that all I had to
do to get the numpy format to work correctly was install numpydoc
above and change:

extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest']

to

extensions = ['sphinx.ext.autodoc', 'sphinx.ext.doctest', 'numpydoc']

in conf.py, so it doesn't seem too hard.

Of course, there's other features in the numpy docstring format that I
haven't demonstrated here, this is just for a simple case. I called
Erik and Prasanth's format "Sphinx-suggested" since it's what the
sphinx docs suggest.

If you have other ideas of docstrings, feel free to fork the repo and
optionally merge back via pull requests!

Cheers,
Tom

On 11 July 2011 18:58, Erik Tollerud <erik.tollerud at gmail.com> wrote:
> I can definitely see the value of the numpy/scipy system, given the
> reasons Joe and Michael have given here.  In particular, the actual
> sectioning/organization of the docstring definitely makes sense for
> our needs and consistency with numpy/scipy is an added bonus.
>
> However, when I thought about it earlier, I saw a number of issues
> with using the full numpy build doc system + pydocweb for the
> particular needs of astropy.  I will say right now that any of these
> may simply be due to my lack of knowledge of how the system works, so
> if there are easily solutions I've missed, I'd love to hear them!
>
> * First, and most importantly, it's not clear to me how to adapt the
> numpy doc builder to our needs.  I've actually never gotten it to run
> completely on numpy, despite trying on three different platforms.  I'm
> willing to concede this could well be user error, but it implies that
> it might be difficult to use on a bunch of new projects - there seem
> to be 8 separate extensions in numpy's "sphinxet". Remember that the
> plan we have set out here involves a large number of packages that
> operate semi-independently until they are merged into the core, and
> they must be able to build their documentation independently of the
> core, at least until they are merged (and many will probably never be
> merged).  Is there a separate project that is documented and can be
> easily adapted/simplified to suit our needs?
>
> * Does the pydocweb system still work since scipy moved to github (or
> in general is it VCS-agnostic)? I have edited scipy docs using the
> system before, but it was never clear to me how the web interface and
> the VCS interplayed with each other.  I would say we want to keep the
> docs themselves in sync with the VCS version of the code they apply to
> - is that always the case with pydocweb?
>
> *The pydocweb workflow diagram requires quite a few steps to
> completion, and as Joe points out, it seems the review system ran out
> of steam - and indeed looking at the numpy docs nearly everything is
> still listed as "need review."  Given that our community is probably
> smaller and has fewer resources to devote to reviewing docs, I'm
> concerned this workflow will not work for astropy (particularly for
> the affiliated packages).  My understanding is that the review system
> was designed basically because there was a bunch of poorly-documented
> functionality, so the idea was to bring other people in to do the
> documentation.  With the vision in place for astropy, nothing even
> makes it into the core in the first place if it doesn't have fairly
> complete documentation.  That's one of the most important jobs of the
> coordinating committee.  So with that in mind, might the full pydocweb
> scheme actually be too *complex* for our needs? Or is it easy to
> simplify it for our needs (perhaps just a single "review" stage that
> comes immediately after the code is written?)
>
> *How does pydocweb handle properties?  We definitely want to be able
> to document properties directly, and have them still end up in the
> sphinx docs alongside the docs for the class' attributes.
>
> *pydocweb requires django to operate, according to it's docs - so that
> implies that either we have to have a central host to hold all of the
> affiliated projects' pydocweb projects while they are being written,
> or we have to expect people to figure that out on their own... By
> contrast, a simpler sphinx setup only requires "python setup.py
> build_docs upload_docs" and then the latest docs are at
> packages.python.org, and fully accessible via PyPI.  Or is there a way
> to bring pydocweb onto packages.python.org ?
>
> Given that sphinx now has tools to generate APIs entirely from
> docstrings (the built-in autodoc and autosummary extensions) without
> any extra scripts needed, might it make more sense to use the numpy
> docstring format, but couple it instead to these sphinx tools? Sphinx
> also has markup specific to all of the sections that the numpy docs
> call for, also, so it's simple to use the ".. seealso::" and "..
> notes" constructs to get the same effect as the numpy docstrings
> without the additional numpy scripts.  This of would mean we'd either
> have to change the syntax of the numpydoc scheme (although hopefully
> not the sectioning), or write a sphinx extension to suit our needs.
>
>
>
>> I think the alternate format discussed by Erik T. and Prasanth is OK,
>> and I've encountered it before (e.g., while reading the AstroLibCoords
>> documentation).  However, I think this format is more cumbersome to
>> read because the use of the additional ":param" or ":returns" at the
>> beginning of each variable definition, and the use of ":type".  For
>> me, these phrases make the text more difficult to quickly search and
>> find the variable name that I would be looking for when using the
>> on-line help.  numpydoc's headings "Parameters", "Returns", etc. is
>> much easier to read.
>
> I agree this is a bit more cumbersome to read, but it produces (at
> least, to my eye) much prettier sphinx APIs if you use it with
> autodoc/autosummary.  So I guess the question is: do we want the
> docstrings as they appear with ipython's "?" or help() to be
> prioritized over the sphinx output, vice-versa, or what?
>
> (Oh, and there's only need for one "returns" clause, and as I was
> saying before, the "type" can be excluded if it's judged to make it
> too hard to read.  The "param name" of course has to be there,
> though).
>
>
>
> On Mon, Jul 11, 2011 at 11:23 AM, Joe Harrington <jh at physics.ucf.edu> wrote:
>> I apologize profusely for replying on the wrong thread!  I've
>> re-posted here so discussion can follow in the right one.  Sorry for
>> the extra traffic.
>>
>>> http://astropy.wikispaces.com/Astropy+Coding+Guidelines
>>
>> I would strongly encourage:
>>
>> 1. Use the numpy docstring format.  We spent a *lot* of time designing
>>   this format and getting community feedback to ensure that projects
>>   like this one can use it, in addition to the core numpy doc
>>   project.  It's been vetted in the writing of over 1000 pages of
>>   docs.  It can handle images, formulae, and cross references and
>>   there are guidelines for how to do that and how to make sure that
>>   the ASCII version is also sensible.  It's what numpy and scipy
>>   users are now used to.  It's based on Sphinx, so it makes a PDF,
>>   HTML, and of course ASCII, has doctests, etc.  There's a place for
>>   references, and a way to separate the basic info everyone needs to
>>   know from the details ("notes") only specialists care about.  There
>>   is one mod that the Matplotlib folks have, for the special case of
>>   families of routines that have dozens or hundreds of identical,
>>   optional parameters (common for graphics routines).
>>
>>   Important: Keep the one-line summary truly to less than about 60
>>   characters.  It can be used to make lists of routines, one line per
>>   routine, including the name, for easy indexing.  This breaks if the
>>   "one line" goes on and on.
>>
>>   Here is the standard:
>>
>>   https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt
>>
>>   If you want to see what the results look like, just type
>>   help(np.whatever), or look at the PDF or HTML pages at
>>   docs.scipy.org.
>>
>> 2. Use the docs.scipy.org wiki system, pydocweb.  Make a project
>>   either there or on the astropy system; the code is OSS.  This will
>>   allow normal users who would not otherwise have access to the code
>>   repo to submit doc fixes, examples, and tests; perform peer review;
>>   and discuss specific pages.  This is extremely helpful if the
>>   programmer is lazy (or even functionally illiterate ;-), or when
>>   importing a package that's already written and has poor docs (e.g.,
>>   numpy 3 years ago).  It can sync in both directions with the VCS,
>>   gives metrics of who contributed what, what needs editing now, what
>>   got worked on this week, etc.  You can see who wrote what, and
>>   revert if there's a problem.  Check out these features at
>>
>>   http://docs.scipy.org/numpy/
>>
>>   A description of the workflow and links to the style guidelines,
>>   templates, and examples are here:
>>
>>   http://docs.scipy.org/numpy/Front%20Page/
>>
>>   Source code to pydocweb is here:
>>
>>   http://code.google.com/p/pydocweb/
>>
>>   The project at numpy is fairly dormant now, the main docs having
>>   been written (the community fell flat on its face and didn't step
>>   up to review), but the folks are all available on the scipy-dev
>>   mailing list if you want to ask questions about it.  The main
>>   developer is Pauli Virtanen.  We wrote status articles in the 2008
>>   and 2009 SciPy Conference proceedings:
>>
>>   http://conference.scipy.org/proceedings/SciPy2008/paper_7/
>>   http://conference.scipy.org/proceedings/SciPy2009/paper_14/
>
>
>
> --
> Erik Tollerud
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>