Re: [Cython] [cython] Autodoc improvements (#216)
Nikita Nemkin, 18.04.2013 07:07:
Three improvements are included in this patch that I have found helpful when using Sphinx autodoc to document my project:
1) Default values preserve and display binary expressions and attribute access. For example, the following function will have its signature displayed as is:
cpdef func(int flags=MyModule.FLAG1 | MyModule.FLAG2)
This only affects expressions not reducible at compilation time, i.e. `2+2` will appear as `4`.
2) Public/readonly cdef attributes can have docstrings attached, using the [PEP 258 attribute docstring syntax](http://www.python.org/dev/peps/pep-0258/#attribute-docstrings). Example:
cdef class X: cdef public attr """attr docstring"""
When ``embedsignature=True`` Cython already adds a signature docstring to the generated property (assuming the example above). This patch allows to add an actual docstring too.
Dosctrings on private cdef attributes will give a warning ("docstring ignored"). Other forms and uses of attribute docstrings, as described in the PEP 258, are not supported at all (SyntaxError), because there is no standard introspection mechanism designed for them. ([PEP 224](http://www.python.org/dev/peps/pep-0224/) describing such mechanism had been rejected.)
3) `bint` appears as `bool` (instead of `int`) in autogenerated signatures, as I suggested on the mailing list.
Each change is contained in its own commit in case you want to review them separately.
I like 1) and 3), but I'm not sure about 2). Worth noting that PEP 258 was rejected, although definitely for other reasons than this feature. I remember discussing this topic before on the mailing list, but don't remember if we reached a conclusion. I faintly recall that we preferred making the property explicit as soon as there's more to it than a plain 1:1 mapping to the cdef attribute. The above syntax is not wrong, but it also doesn't feel right to me. For example, it's impossible to draw a distinction between the above and something like cdef class X: cdef public attr """ quotes used for commenting out code here. """ And users are unlikely to notice that they just dropped some internals into the docstring of a property they didn't even know they were implementing. IMHO, the user intention is way too unclear in this syntactic construct. Stefan
On Sat, 20 Apr 2013 02:11:21 +0600, Stefan Behnel <stefan_ml@behnel.de> wrote:
Nikita Nemkin, 18.04.2013 07:07:
2) Public/readonly cdef attributes can have docstrings attached, using the [PEP 258 attribute docstring syntax](http://www.python.org/dev/peps/pep-0258/#attribute-docstrings). Example:
cdef class X: cdef public attr """attr docstring"""
When ``embedsignature=True`` Cython already adds a signature docstring to the generated property (assuming the example above). This patch allows to add an actual docstring too.
Dosctrings on private cdef attributes will give a warning ("docstring ignored"). Other forms and uses of attribute docstrings, as described in the PEP 258, are not supported at all (SyntaxError), because there is no standard introspection mechanism designed for them. ([PEP 224](http://www.python.org/dev/peps/pep-0224/) describing such mechanism had been rejected.)
Worth noting that PEP 258 was rejected, although definitely for other reasons than this feature. I remember discussing this topic before on the mailing list, but don't remember if we reached a conclusion. I faintly recall that we preferred making the property explicit as soon as there's more to it than a plain 1:1 mapping to the cdef attribute.
Separating property from the attribute requires mass renaming of the attribute (as was recently descussed here) and bloating the code x4. All this just to attach a docstring? PEP 258 was rejected, but Sphinx supports it, an so does epydoc. The need to autodocument attributes remains.
The above syntax is not wrong, but it also doesn't feel right to me. For example, it's impossible to draw a distinction between the above and something like
cdef class X: cdef public attr
""" quotes used for commenting out code here. """
And users are unlikely to notice that they just dropped some internals into the docstring of a property they didn't even know they were implementing.
IMHO, the user intention is way too unclear in this syntactic construct.
Attribute docstring must appear immediately after the attribute. The extra blank line in your example makes the string standalone, just as you intended. Also, internal comments in the middle of the class are not usually put into strings, actual comment syntax is more appropriate here. Documenting attributes manually, for example, with ".. attribute" directives in the class docstring or .rst file, creates the following problems: 1) In embedsignature mode (which is almost mandatory for a decent autodoc) Cython adds signature docstrings to public attributes, making them appear twice: one instance with a manually written doc, another instance with a signature docstring. 2) Manually documented attributes don't follow autodoc settings (ordering, formatting). They are also invisible to autosummary and other extensions. 3) (Obviously) non-uniformity between property, function and attribute documentation practice. Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.) Best regards, Nikita Nemkin
Nikita Nemkin, 20.04.2013 09:11:
On Sat, 20 Apr 2013 02:11:21 +0600, Stefan Behnel wrote:
Nikita Nemkin, 18.04.2013 07:07:
2) Public/readonly cdef attributes can have docstrings attached, using the [PEP 258 attribute docstring syntax](http://www.python.org/dev/peps/pep-0258/#attribute-docstrings). Example:
cdef class X: cdef public attr """attr docstring"""
When ``embedsignature=True`` Cython already adds a signature docstring to the generated property (assuming the example above). This patch allows to add an actual docstring too.
Dosctrings on private cdef attributes will give a warning ("docstring ignored"). Other forms and uses of attribute docstrings, as described in the PEP 258, are not supported at all (SyntaxError), because there is no standard introspection mechanism designed for them. ([PEP 224](http://www.python.org/dev/peps/pep-0224/) describing such mechanism had been rejected.)
Worth noting that PEP 258 was rejected, although definitely for other reasons than this feature. I remember discussing this topic before on the mailing list, but don't remember if we reached a conclusion. I faintly recall that we preferred making the property explicit as soon as there's more to it than a plain 1:1 mapping to the cdef attribute.
Separating property from the attribute requires mass renaming of the attribute (as was recently descussed here) and bloating the code x4. All this just to attach a docstring?
Well, once there is a docstring, that factor goes down to at most 3x. It's not entirely overkill, although I agree that it's definitely way more complexity than just adding a docstring.
PEP 258 was rejected, but Sphinx supports it, an so does epydoc.
That makes it more reasonable then, but see below.
The above syntax is not wrong, but it also doesn't feel right to me. For example, it's impossible to draw a distinction between the above and something like
cdef class X: cdef public attr
""" quotes used for commenting out code here. """
And users are unlikely to notice that they just dropped some internals into the docstring of a property they didn't even know they were implementing.
IMHO, the user intention is way too unclear in this syntactic construct.
Attribute docstring must appear immediately after the attribute. The extra blank line in your example makes the string standalone, just as you intended.
I can't see that from the tests you wrote.
Also, internal comments in the middle of the class are not usually put into strings, actual comment syntax is more appropriate here.
The less common it is, the more likely it is to be a source of confusion when it happens. And the proposed syntax is definitely prone to errors that go undetected.
Documenting attributes manually, for example, with ".. attribute" directives in the class docstring or .rst file, creates the following problems: 1) In embedsignature mode (which is almost mandatory for a decent autodoc)
It's not more than a clumsy work-around for the CPython quirk that C implemented functions couldn't expose their signature. And that problem is much better tackled by completing the support for PEP 362. http://www.python.org/dev/peps/pep-0362/
1) In embedsignature mode (which is almost mandatory for a decent autodoc) Cython adds signature docstrings to public attributes, making them appear twice: one instance with a manually written doc, another instance with a signature docstring.
2) Manually documented attributes don't follow autodoc settings (ordering, formatting). They are also invisible to autosummary and other extensions.
3) (Obviously) non-uniformity between property, function and attribute documentation practice.
The way I see it, if the embedsignature feature actually died because of PEP 362, you could just document attributes yourself without having Cython interfere with it at all. And AFAIU, that would solve most of your above problems.
Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.)
It's not about the implementation. The question is if this feature should become part of the language because it's "special enough to break the rules" of OOTDI and if so, how we should expose it. And yes, that's a wonderful field for bike shedding. Stefan
On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel <stefan_ml@behnel.de> wrote:
Nikita Nemkin, 20.04.2013 09:11:
On Sat, 20 Apr 2013 02:11:21 +0600, Stefan Behnel wrote: Separating property from the attribute requires mass renaming of the attribute (as was recently descussed here) and bloating the code x4. All this just to attach a docstring?
Well, once there is a docstring, that factor goes down to at most 3x. It's not entirely overkill, although I agree that it's definitely way more complexity than just adding a docstring.
PEP 258 was rejected, but Sphinx supports it, an so does epydoc.
That makes it more reasonable then, but see below.
The above syntax is not wrong, but it also doesn't feel right to me. For example, it's impossible to draw a distinction between the above and something like
cdef class X: cdef public attr
""" quotes used for commenting out code here. """
And users are unlikely to notice that they just dropped some internals into the docstring of a property they didn't even know they were implementing.
IMHO, the user intention is way too unclear in this syntactic construct.
Attribute docstring must appear immediately after the attribute. The extra blank line in your example makes the string standalone, just as you intended.
I can't see that from the tests you wrote.
I honestly didn't expect random string literals in the middle of the class. Adding an extra test is never a problem.
Also, internal comments in the middle of the class are not usually put into strings, actual comment syntax is more appropriate here.
The less common it is, the more likely it is to be a source of confusion when it happens. And the proposed syntax is definitely prone to errors that go undetected.
Very unlikely and very minor documentation(!) errors. Wrong docstring on a property descriptor, couldn't be any more harmless. I don't think "bad syntax leads to errors" argument holds here.
Documenting attributes manually, for example, with ".. attribute" directives in the class docstring or .rst file, creates the following problems: 1) In embedsignature mode (which is almost mandatory for a decent autodoc)
It's not more than a clumsy work-around for the CPython quirk that C implemented functions couldn't expose their signature. And that problem is much better tackled by completing the support for PEP 362.
PEP362 is irrelevant for properties and attributes. Anyway, I was just listing technicals problems biting autodoc users.
1) In embedsignature mode (which is almost mandatory for a decent autodoc) Cython adds signature docstrings to public attributes, making them appear twice: one instance with a manually written doc, another instance with a signature docstring.
2) Manually documented attributes don't follow autodoc settings (ordering, formatting). They are also invisible to autosummary and other extensions.
3) (Obviously) non-uniformity between property, function and attribute documentation practice.
The way I see it, if the embedsignature feature actually died because of PEP 362, you could just document attributes yourself without having Cython interfere with it at all. And AFAIU, that would solve most of your above problems.
I did document them manually at first. It is ugly (function doc in one place, attribute doc in another) and it does not work well with autodoc and autosummary (no summaries for attributes, attrbute section appears in the wrong place and other "minor" problems that require heavy Sphinx hacks to get around.) All in all, it completely defeats the purpose of AUTOdoc. The practice of autodocumentation is widespread in Python (docstrings are a part of the language semantics) and I'd rather it support properties and attributes without messy workarounds. Sphinx has some way to go in this area, but putting strings into __doc__ slots is Cython's responsibility.
Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.)
It's not about the implementation. The question is if this feature should become part of the language because it's "special enough to break the rules" of OOTDI and if so, how we should expose it. And yes, that's a wonderful field for bike shedding.
It does not change the language syntax one bit, it just adds a semantic on top, like all docstrings do. Think of it this way: cdef public/readonly is a shorthand for a complete property declaration. Similarly, attribute docstring is a shortcut for providing a __doc__ for this property. PEP258/Sphinx/epydoc already have an established convention for such docstrings, we just put these pieces together. A mention that a docstring can be provided for a cdef public attribute is all "exposure" it needs. Those who don't need/don't know about it won't be affected. Best regards, Nikita Nemkin
Nikita Nemkin, 20.04.2013 12:43:
On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel wrote:
Nikita Nemkin, 20.04.2013 09:11:
On Sat, 20 Apr 2013 02:11:21 +0600, Stefan Behnel wrote: Separating property from the attribute requires mass renaming of the attribute (as was recently descussed here) and bloating the code x4. All this just to attach a docstring?
Well, once there is a docstring, that factor goes down to at most 3x. It's not entirely overkill, although I agree that it's definitely way more complexity than just adding a docstring.
PEP 258 was rejected, but Sphinx supports it, an so does epydoc.
That makes it more reasonable then, but see below.
The above syntax is not wrong, but it also doesn't feel right to me. For example, it's impossible to draw a distinction between the above and something like
cdef class X: cdef public attr
""" quotes used for commenting out code here. """
And users are unlikely to notice that they just dropped some internals into the docstring of a property they didn't even know they were implementing.
IMHO, the user intention is way too unclear in this syntactic construct.
Attribute docstring must appear immediately after the attribute. The extra blank line in your example makes the string standalone, just as you intended.
I can't see that from the tests you wrote.
I honestly didn't expect random string literals in the middle of the class. Adding an extra test is never a problem.
I was just asking for a proof of your above statement. My experience with Cython's parser doesn't indicate per se that this is the case.
Also, internal comments in the middle of the class are not usually put into strings, actual comment syntax is more appropriate here.
The less common it is, the more likely it is to be a source of confusion when it happens. And the proposed syntax is definitely prone to errors that go undetected.
Very unlikely and very minor documentation(!) errors. Wrong docstring on a property descriptor, couldn't be any more harmless. I don't think "bad syntax leads to errors" argument holds here.
So let's agree to disagree on this.
Documenting attributes manually, for example, with ".. attribute" directives in the class docstring or .rst file, creates the following problems: 1) In embedsignature mode (which is almost mandatory for a decent autodoc)
It's not more than a clumsy work-around for the CPython quirk that C implemented functions couldn't expose their signature. And that problem is much better tackled by completing the support for PEP 362.
PEP362 is irrelevant for properties and attributes.
Yep, different things. That was part of my point.
1) In embedsignature mode (which is almost mandatory for a decent autodoc) Cython adds signature docstrings to public attributes, making them appear twice: one instance with a manually written doc, another instance with a signature docstring.
2) Manually documented attributes don't follow autodoc settings (ordering, formatting). They are also invisible to autosummary and other extensions.
3) (Obviously) non-uniformity between property, function and attribute documentation practice.
The way I see it, if the embedsignature feature actually died because of PEP 362, you could just document attributes yourself without having Cython interfere with it at all. And AFAIU, that would solve most of your above problems.
I did document them manually at first. It is ugly (function doc in one place, attribute doc in another)
As it should be. The attributes are part of the class documentation, whereas the function/method documentation belongs to the functions. I can't see why you consider this totally normal distinction "ugly".
and it does not work well with autodoc and autosummary (no summaries for attributes, attrbute section appears in the wrong place and other "minor" problems that require heavy Sphinx hacks to get around.) All in all, it completely defeats the purpose of AUTOdoc.
The practice of autodocumentation is widespread in Python (docstrings are a part of the language semantics) and I'd rather it support properties and attributes without messy workarounds. Sphinx has some way to go in this area, but putting strings into __doc__ slots is Cython's responsibility.
All I'm saying is that docstrings for cdef attributes are a very special case. If you had a Python class in a Cython module, you couldn't add a docstring to its attributes at all. And there already is a dedicated syntax for non-trivial properties in cdef classes. So the question is: is a property with a docstring still a trivial property or not? And is the use case common enough to complicate the simple special case that already exists, even though the completely general case can already be handled?
Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.)
It's not about the implementation. The question is if this feature should become part of the language because it's "special enough to break the rules" of OOTDI and if so, how we should expose it. And yes, that's a wonderful field for bike shedding.
It does not change the language syntax one bit
I didn't say it would change the syntax. It changes the semantics of a string that follows an attribute declaration (for whatever reason) and makes strings that appear after such declarations the official One Right Way to document cdef attributes, i.e. a Cython language feature. BTW, the pure Python mode doesn't currently have this feature either, it seems. (I'm not even sure it has public/readonly attributes...)
cdef public/readonly is a shorthand for a complete property declaration.
Sure, for the trivial case.
Similarly, attribute docstring is a shortcut for providing a __doc__ for this property. PEP258/Sphinx/epydoc already have an established convention for such docstrings, we just put these pieces together.
ISTM that there is more than one convention: http://sphinx-doc.org/ext/autodoc.html#directive-autoattribute But the newer one apparently uses the docstring syntax (which IMHO makes way more sense than special casing comments).
A mention that a docstring can be provided for a cdef public attribute is all "exposure" it needs. Those who don't need/don't know about it won't be affected.
Except by accident, as I said. Users of Sphinx *may* be aware of this, others most likely won't be. Stefan
On Sat, 20 Apr 2013 17:41:56 +0600, Stefan Behnel <stefan_ml@behnel.de> wrote:
Nikita Nemkin, 20.04.2013 12:43:
On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel wrote: I did document them manually at first. It is ugly (function doc in one place, attribute doc in another)
As it should be. The attributes are part of the class documentation, whereas the function/method documentation belongs to the functions. I can't see why you consider this totally normal distinction "ugly".
Why should attributes and especially properties (class members) be treated differently from methods (class members) for documentation purposes? From the OOP standpoint they certainly should not. Smalltalk and Java are good examples. (Smalltalk even has a documentation convention extremely similar to Python docstrings.)
All I'm saying is that docstrings for cdef attributes are a very special case. If you had a Python class in a Cython module, you couldn't add a docstring to its attributes at all. And there already is a dedicated syntax for non-trivial properties in cdef classes. So the question is: is a property with a docstring still a trivial property or not? And is the use case common enough to complicate the simple special case that already exists, even though the completely general case can already be handled?
"cdef public" is one of the two forms of property declaration in Cython. (The other being "property: ...") Both use nonstandard Python syntax and neither seems a "special case" to me. The property is trivial iif its code consists of one __get__ function with "return self.value" body. What does docstring have to do with it? "property:" is not completely general. I wont rename every single public attribute for the privilege of attaching a docstring to it (and a ton of boilerplate code). It makes no sense from technical and maintenance standpoint. Considering use cases. Currently I have 83 cdef public/readonly attributes across 46 classes that represent ~20% of the source codebase. All of them have docstrings (since they are public and part of the API.) This feature is very low impact to worry about added complexity.
Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.)
It's not about the implementation. The question is if this feature should become part of the language because it's "special enough to break the rules" of OOTDI and if so, how we should expose it. And yes, that's a wonderful field for bike shedding.
It does not change the language syntax one bit
I didn't say it would change the syntax. It changes the semantics of a string that follows an attribute declaration (for whatever reason) and makes strings that appear after such declarations the official One Right Way to document cdef attributes, i.e. a Cython language feature.
Since there are currently Zero Right Ways to document these attributes, having at least one is a good thing. "property:" form is not diminished by that, in cases _when it is warranted_.
BTW, the pure Python mode doesn't currently have this feature either, it seems. (I'm not even sure it has public/readonly attributes...)
Pure python mode project can be processed by Sphinx like any ordinary Python project, with complete support for all autodoc features. Since Python level attributes have no __doc__ slots, there is nothing Cython can do with their docstrings anyway (except parsing them without syntax errors).
Similarly, attribute docstring is a shortcut for providing a __doc__ for this property. PEP258/Sphinx/epydoc already have an established convention for such docstrings, we just put these pieces together.
ISTM that there is more than one convention:
http://sphinx-doc.org/ext/autodoc.html#directive-autoattribute
Other conventions? Do you mean "#:" comments? These forms are not part of PEP258, can't be paralleled to the "property:" syntax and require too many changes to Cython lexer and parser for no extra benefit. Therefore I have not implemented them.
But the newer one apparently uses the docstring syntax (which IMHO makes way more sense than special casing comments).
Exactly.
A mention that a docstring can be provided for a cdef public attribute is all "exposure" it needs. Those who don't need/don't know about it won't be affected.
Except by accident, as I said. Users of Sphinx *may* be aware of this, others most likely won't be.
In the worst case, probability of which is near zero, the unsuspecting victim will waste a few bytes of memory (the size of the comment). This is a non-issue. Best regards, Nikita Nemkin
Nikita Nemkin, 20.04.2013 16:44:
On Sat, 20 Apr 2013 17:41:56 +0600, Stefan Behnel wrote:
Nikita Nemkin, 20.04.2013 12:43:
On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel wrote: I did document them manually at first. It is ugly (function doc in one place, attribute doc in another)
As it should be. The attributes are part of the class documentation, whereas the function/method documentation belongs to the functions. I can't see why you consider this totally normal distinction "ugly".
Why should attributes and especially properties (class members) be treated differently from methods (class members) for documentation purposes? From the OOP standpoint they certainly should not. Smalltalk and Java are good examples. (Smalltalk even has a documentation convention extremely similar to Python docstrings.)
Just because some languages don't have properties and always require you to implement getters and setters doesn't mean that attributes, methods and properties are the same thing. The question whether an attribute is implemented as a field or a property is an implementation detail that shouldn't have an impact on users. That makes attributes different from methods. Thus my comment that attributes should be part of the class documentation. That way, you can document them consistently for both extension types and Python classes. Because in the best case, users shouldn't have to care about that distinction either. And with PEP 362 in place, there'll no longer be a need for automatically embedded signatures etc. either. Attributes can be documented as part of the class and methods document themselves anyway.
All I'm saying is that docstrings for cdef attributes are a very special case. If you had a Python class in a Cython module, you couldn't add a docstring to its attributes at all. And there already is a dedicated syntax for non-trivial properties in cdef classes. So the question is: is a property with a docstring still a trivial property or not? And is the use case common enough to complicate the simple special case that already exists, even though the completely general case can already be handled?
"cdef public" is one of the two forms of property declaration in Cython. (The other being "property: ...") Both use nonstandard Python syntax and neither seems a "special case" to me.
"cdef public" is the special case because it's pure syntactic sugar that is redundant with the most simple application of the property syntax.
The property is trivial iif its code consists of one __get__ function with "return self.value" body. What does docstring have to do with it?
The general way of doing it (i.e. the property syntax) already has a way of specifying a docstring. That's a reason to be hesitant about adding this feature also to the (currently) simple short-hand special case syntax.
"property:" is not completely general. I wont rename every single public attribute for the privilege of attaching a docstring to it (and a ton of boilerplate code). It makes no sense from technical and maintenance standpoint.
I agree that it might seem overkill.
Considering use cases. Currently I have 83 cdef public/readonly attributes across 46 classes that represent ~20% of the source codebase. All of them have docstrings (since they are public and part of the API.)
That seems a lot to me. I wonder how common this actually is. In the code I've written so far, the trivial case of 1:1 properties was pretty rare. Might be a domain thing. Stefan
Jumping into this thread late, improvements (1) and (3) certainly seem beneficial. As far as documenting "attributes," I can't see much of a use, but the downsides seem are low enough (accidentally assigning a stray string to the previously declared attribute) and the implementation non-invasive enough that I'm +0.5 for this change. It doesn't introduce new syntax or behavior, just additional semantic meaning for those (otherwise unlikely to occur) strings and sure beats writing them out as properties (though the fact that cdef attributes are properties should be considered more of an implementation detail IMHO). - Robert On Sat, Apr 20, 2013 at 7:44 AM, Nikita Nemkin <nikita@nemkin.ru> wrote:
On Sat, 20 Apr 2013 17:41:56 +0600, Stefan Behnel <stefan_ml@behnel.de> wrote:
Nikita Nemkin, 20.04.2013 12:43:
On Sat, 20 Apr 2013 14:27:12 +0600, Stefan Behnel wrote: I did document them manually at first. It is ugly (function doc in one place, attribute doc in another)
As it should be. The attributes are part of the class documentation, whereas the function/method documentation belongs to the functions. I can't see why you consider this totally normal distinction "ugly".
Why should attributes and especially properties (class members) be treated differently from methods (class members) for documentation purposes? From the OOP standpoint they certainly should not. Smalltalk and Java are good examples. (Smalltalk even has a documentation convention extremely similar to Python docstrings.)
All I'm saying is that docstrings for cdef attributes are a very special case. If you had a Python class in a Cython module, you couldn't add a docstring to its attributes at all. And there already is a dedicated syntax for non-trivial properties in cdef classes. So the question is: is a property with a docstring still a trivial property or not? And is the use case common enough to complicate the simple special case that already exists, even though the completely general case can already be handled?
"cdef public" is one of the two forms of property declaration in Cython. (The other being "property: ...") Both use nonstandard Python syntax and neither seems a "special case" to me.
The property is trivial iif its code consists of one __get__ function with "return self.value" body. What does docstring have to do with it? "property:" is not completely general. I wont rename every single public attribute for the privilege of attaching a docstring to it (and a ton of boilerplate code). It makes no sense from technical and maintenance standpoint.
Considering use cases. Currently I have 83 cdef public/readonly attributes across 46 classes that represent ~20% of the source codebase. All of them have docstrings (since they are public and part of the API.)
This feature is very low impact to worry about added complexity.
Attribute docstrings are the cleanest solution to all of these problems. And the patch is just a few lines (if you discount the refactoring of the "XXX This should go to AutoDocTransforms" block.)
It's not about the implementation. The question is if this feature should become part of the language because it's "special enough to break the rules" of OOTDI and if so, how we should expose it. And yes, that's a wonderful field for bike shedding.
It does not change the language syntax one bit
I didn't say it would change the syntax. It changes the semantics of a string that follows an attribute declaration (for whatever reason) and makes strings that appear after such declarations the official One Right Way to document cdef attributes, i.e. a Cython language feature.
Since there are currently Zero Right Ways to document these attributes, having at least one is a good thing. "property:" form is not diminished by that, in cases _when it is warranted_.
BTW, the pure Python mode doesn't currently have this feature either, it seems. (I'm not even sure it has public/readonly attributes...)
Pure python mode project can be processed by Sphinx like any ordinary Python project, with complete support for all autodoc features.
Since Python level attributes have no __doc__ slots, there is nothing Cython can do with their docstrings anyway (except parsing them without syntax errors).
Similarly, attribute docstring is a shortcut for providing a __doc__ for this property. PEP258/Sphinx/epydoc already have an established convention for such docstrings, we just put these pieces together.
ISTM that there is more than one convention:
http://sphinx-doc.org/ext/autodoc.html#directive-autoattribute
Other conventions? Do you mean "#:" comments? These forms are not part of PEP258, can't be paralleled to the "property:" syntax and require too many changes to Cython lexer and parser for no extra benefit. Therefore I have not implemented them.
But the newer one apparently uses the docstring syntax (which IMHO makes way more sense than special casing comments).
Exactly.
A mention that a docstring can be provided for a cdef public attribute is all "exposure" it needs. Those who don't need/don't know about it won't be affected.
Except by accident, as I said. Users of Sphinx *may* be aware of this, others most likely won't be.
In the worst case, probability of which is near zero, the unsuspecting victim will waste a few bytes of memory (the size of the comment). This is a non-issue.
Best regards, Nikita Nemkin
_______________________________________________ cython-devel mailing list cython-devel@python.org http://mail.python.org/mailman/listinfo/cython-devel
On Tue, 23 Apr 2013 10:19:15 +0600, Robert Bradshaw <robertwb@gmail.com> wrote:
Jumping into this thread late, improvements (1) and (3) certainly seem beneficial. As far as documenting "attributes," I can't see much of a use, but the downsides seem are low enough (accidentally assigning a stray string to the previously declared attribute) and the implementation non-invasive enough that I'm +0.5 for this change. It doesn't introduce new syntax or behavior, just additional semantic meaning for those (otherwise unlikely to occur) strings and sure beats writing them out as properties (though the fact that cdef attributes are properties should be considered more of an implementation detail IMHO).
I've just found a 4 year old ticket requesting the same feature (2): http://trac.cython.org/cython_trac/ticket/206 Best regards, Nikita Nemkin
Nikita Nemkin, 26.04.2013 13:33:
On Tue, 23 Apr 2013 10:19:15 +0600, Robert Bradshaw wrote:
Jumping into this thread late, improvements (1) and (3) certainly seem beneficial. As far as documenting "attributes," I can't see much of a use, but the downsides seem are low enough (accidentally assigning a stray string to the previously declared attribute) and the implementation non-invasive enough that I'm +0.5 for this change. It doesn't introduce new syntax or behavior, just additional semantic meaning for those (otherwise unlikely to occur) strings and sure beats writing them out as properties (though the fact that cdef attributes are properties should be considered more of an implementation detail IMHO).
I've just found a 4 year old ticket requesting the same feature (2): http://trac.cython.org/cython_trac/ticket/206
Just because there's a ticket doesn't mean that it should be fixed, but I'll give in on this one. Nikita, if you add the test that proves that newlines prevent a subsequent string from becoming a docstring, i.e. that cdef attr "this is not a docstring" behaves as expected and is different from cdef attr "this is a docstring" I'll merge your pull request. Stefan
Stefan Behnel, 01.05.2013 11:50:
Nikita Nemkin, 26.04.2013 13:33:
On Tue, 23 Apr 2013 10:19:15 +0600, Robert Bradshaw wrote:
Jumping into this thread late, improvements (1) and (3) certainly seem beneficial. As far as documenting "attributes," I can't see much of a use, but the downsides seem are low enough (accidentally assigning a stray string to the previously declared attribute) and the implementation non-invasive enough that I'm +0.5 for this change. It doesn't introduce new syntax or behavior, just additional semantic meaning for those (otherwise unlikely to occur) strings and sure beats writing them out as properties (though the fact that cdef attributes are properties should be considered more of an implementation detail IMHO).
I've just found a 4 year old ticket requesting the same feature (2): http://trac.cython.org/cython_trac/ticket/206
Just because there's a ticket doesn't mean that it should be fixed, but I'll give in on this one. Nikita, if you add the test that proves that newlines prevent a subsequent string from becoming a docstring, i.e. that
cdef attr
"this is not a docstring"
behaves as expected and is different from
cdef attr "this is a docstring"
I'll merge your pull request.
... and I just noticed that you already did that, and also removed the warning about stray docstrings. Sorry for that. I'll merge it in. Stefan
participants (3)
-
Nikita Nemkin -
Robert Bradshaw -
Stefan Behnel