[Cython] Autodoc improvements (#216)

[Stefan Behnel]

Nikita Nemkin, 20.04.2013 12:43:
> On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel wrote:
>> Nikita Nemkin, 20.04.2013 09:11:
>>> On Sat, 20 Apr 2013 02:11:21 +0600, Stefan Behnel wrote:
>>> Separating property from the attribute requires mass renaming of
>>> the attribute (as was recently descussed here) and bloating the code x4.
>>> All this just to attach a docstring?
>>
>> Well, once there is a docstring, that factor goes down to at most 3x. It's
>> not entirely overkill, although I agree that it's definitely way more
>> complexity than just adding a docstring.
>>
>>> PEP 258 was rejected, but Sphinx supports it, an so does epydoc.
>>
>> That makes it more reasonable then, but see below.
>>
>>>> The above syntax is not wrong, but it also doesn't feel right to me. For
>>>> example, it's impossible to draw a distinction between the above and
>>>> something like
>>>>
>>>>      cdef class X:
>>>>          cdef public attr
>>>>
>>>>          """
>>>>          quotes used for commenting out code here.
>>>>          """
>>>>
>>>> And users are unlikely to notice that they just dropped some internals
>>>> into
>>>> the docstring of a property they didn't even know they were implementing.
>>>>
>>>> IMHO, the user intention is way too unclear in this syntactic construct.
>>>
>>> Attribute docstring must appear immediately after the attribute. The extra
>>> blank line in your example makes the string standalone, just as you
>>> intended.
>>
>> I can't see that from the tests you wrote.
> 
> I honestly didn't expect random string literals in the middle of
> the class. Adding an extra test is never a problem.

I was just asking for a proof of your above statement. My experience with
Cython's parser doesn't indicate per se that this is the case.


>>> Also, internal comments in the middle of the class are not usually
>>> put into strings, actual comment syntax is more appropriate here.
>>
>> The less common it is, the more likely it is to be a source of confusion
>> when it happens. And the proposed syntax is definitely prone to errors
>> that go undetected.
> 
> Very unlikely and very minor documentation(!) errors.
> Wrong docstring on a property descriptor, couldn't be any more harmless.
> I don't think "bad syntax leads to errors" argument holds here.

So let's agree to disagree on this.


>>> Documenting attributes manually, for example, with ".. attribute"
>>> directives in the class docstring or .rst file, creates the following
>>> problems:
>>> 1) In embedsignature mode (which is almost mandatory for a decent autodoc)
>>
>> It's not more than a clumsy work-around for the CPython quirk that C
>> implemented functions couldn't expose their signature. And that problem is
>> much better tackled by completing the support for PEP 362.
>>
>> http://www.python.org/dev/peps/pep-0362/
> 
> PEP362 is irrelevant for properties and attributes.

Yep, different things. That was part of my point.


>>> 1) In embedsignature mode (which is almost mandatory for a decent
>>> autodoc) Cython adds signature docstrings to public attributes, making
>>> them appear twice: one instance with a manually written doc, another
>>> instance with a signature docstring.
>>>
>>> 2) Manually documented attributes don't follow autodoc settings
>>> (ordering, formatting). They are also invisible to autosummary and other
>>> extensions.
>>>
>>> 3) (Obviously) non-uniformity between property, function and attribute
>>> documentation practice.
>>
>> The way I see it, if the embedsignature feature actually died because of
>> PEP 362, you could just document attributes yourself without having Cython
>> interfere with it at all. And AFAIU, that would solve most of your above
>> problems.
> 
> I did document them manually at first. It is ugly (function doc in one place,
> attribute doc in another)

As it should be. The attributes are part of the class documentation,
whereas the function/method documentation belongs to the functions. I can't
see why you consider this totally normal distinction "ugly".


> and it does not work well with autodoc and
> autosummary (no summaries for attributes, attrbute section appears in the
> wrong place and other "minor" problems that require heavy Sphinx hacks to
> get around.) All in all, it completely defeats the purpose of AUTOdoc.
> 
> The practice of autodocumentation is widespread in Python
> (docstrings are a part of the language semantics) and I'd rather it
> support properties and attributes without messy workarounds.
> Sphinx has some way to go in this area, but putting strings
> into __doc__ slots is Cython's responsibility.

All I'm saying is that docstrings for cdef attributes are a very special
case. If you had a Python class in a Cython module, you couldn't add a
docstring to its attributes at all. And there already is a dedicated syntax
for non-trivial properties in cdef classes. So the question is: is a
property with a docstring still a trivial property or not? And is the use
case common enough to complicate the simple special case that already
exists, even though the completely general case can already be handled?


>>> Attribute docstrings are the cleanest solution to all of these problems.
>>> And the patch is just a few lines (if you discount the refactoring of
>>> the "XXX This should go to AutoDocTransforms" block.)
>>
>> It's not about the implementation. The question is if this feature should
>> become part of the language because it's "special enough to break the
>> rules" of OOTDI and if so, how we should expose it. And yes, that's a
>> wonderful field for bike shedding.
> 
> It does not change the language syntax one bit

I didn't say it would change the syntax. It changes the semantics of a
string that follows an attribute declaration (for whatever reason) and
makes strings that appear after such declarations the official One Right
Way to document cdef attributes, i.e. a Cython language feature.

BTW, the pure Python mode doesn't currently have this feature either, it
seems. (I'm not even sure it has public/readonly attributes...)


> cdef public/readonly is a shorthand for a complete property declaration.

Sure, for the trivial case.


> Similarly, attribute docstring is a shortcut for providing a __doc__
> for this property. PEP258/Sphinx/epydoc already have an established
> convention for such docstrings, we just put these pieces together.

ISTM that there is more than one convention:

http://sphinx-doc.org/ext/autodoc.html#directive-autoattribute

But the newer one apparently uses the docstring syntax (which IMHO makes
way more sense than special casing comments).


> A mention that a docstring can be provided for a cdef public attribute
> is all "exposure" it needs. Those who don't need/don't know about it
> won't be affected.

Except by accident, as I said. Users of Sphinx *may* be aware of this,
others most likely won't be.

Stefan