Best way to specify docstrings for member objects
I'm working on ways to make improve help() by giving docstrings to member objects. One way to do it is to wait until after the class definition and then make individual, direct assignments to __doc__ attributes.This way widely the separates docstrings from their initial __slots__ definition. Working downstream from the class definition feels awkward and doesn't look pretty. There's another way I would like to propose¹. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings. This keeps all the relevant information in one place (much like we already do with property() objects). This way already works, we just need a few lines in pydoc to check to see if a dict if present. This way also looks pretty and doesn't feel awkward. I've included worked out examples below. What do you all think about the proposal? Raymond ¹ https://bugs.python.org/issue36326 ====== Desired help() output ======
help(NormalDist) Help on class NormalDist in module __main__:
class NormalDist(builtins.object) | NormalDist(mu=0.0, sigma=1.0) | | Normal distribution of a random variable | | Methods defined here: | | __init__(self, mu=0.0, sigma=1.0) | NormalDist where mu is the mean and sigma is the standard deviation. | | cdf(self, x) | Cumulative distribution function. P(X <= x) | | pdf(self, x) | Probability density function. P(x <= X < x+dx) / dx | | ---------------------------------------------------------------------- | Data descriptors defined here: | | mu | Arithmetic mean. | | sigma | Standard deviation. | | variance | Square of the standard deviation. ====== Example of assigning docstrings after the class definition ====== class NormalDist: 'Normal distribution of a random variable' __slots__ = ('mu', 'sigma') def __init__(self, mu=0.0, sigma=1.0): 'NormalDist where mu is the mean and sigma is the standard deviation.' self.mu = mu self.sigma = sigma @property def variance(self): 'Square of the standard deviation.' return self.sigma ** 2. def pdf(self, x): 'Probability density function. P(x <= X < x+dx) / dx' variance = self.variance return exp((x - self.mu)**2.0 / (-2.0*variance)) / sqrt(tau * variance) def cdf(self, x): 'Cumulative distribution function. P(X <= x)' return 0.5 * (1.0 + erf((x - self.mu) / (self.sigma * sqrt(2.0)))) NormalDist.mu.__doc__ = 'Arithmetic mean' NormalDist.sigma.__doc__ = 'Standard deviation' ====== Example of assigning docstrings with a dict ===== class NormalDist: 'Normal distribution of a random variable' __slots__ = {'mu' : 'Arithmetic mean.', 'sigma': 'Standard deviation.'} def __init__(self, mu=0.0, sigma=1.0): 'NormalDist where mu is the mean and sigma is the standard deviation.' self.mu = mu self.sigma = sigma @property def variance(self): 'Square of the standard deviation.' return self.sigma ** 2. def pdf(self, x): 'Probability density function. P(x <= X < x+dx) / dx' variance = self.variance return exp((x - self.mu)**2.0 / (-2.0*variance)) / sqrt(tau * variance) def cdf(self, x): 'Cumulative distribution function. P(X <= x)' return 0.5 * (1.0 + erf((x - self.mu) / (self.sigma * sqrt(2.0))))
I have the impression that the line between variables and docs is a tidbit too much blurred. Yours, Abdur-Rahmaan Janhangeer Mauritius
On 2019-03-19 18:55, Raymond Hettinger wrote:
I'm working on ways to make improve help() by giving docstrings to member objects.
One way to do it is to wait until after the class definition and then make individual, direct assignments to __doc__ attributes.This way widely the separates docstrings from their initial __slots__ definition. Working downstream from the class definition feels awkward and doesn't look pretty.
There's another way I would like to propose¹. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
This keeps all the relevant information in one place (much like we already do with property() objects). This way already works, we just need a few lines in pydoc to check to see if a dict if present. This way also looks pretty and doesn't feel awkward.
I've included worked out examples below. What do you all think about the proposal?
[snip] Thinking ahead, could there ever be anything else that you might want also to attach to member objects? I suppose that if that's ever the case, the value could itself be expanded to be a dict, something like this: __slots__ = {'mu' : {'__doc__': 'Arithmetic mean.'}, 'sigma': {'__doc__': 'Standard deviation.'}} But that could be left to the future...
On Mar 19, 2019, at 1:52 PM, MRAB
wrote: Thinking ahead, could there ever be anything else that you might want also to attach to member objects?
Our experience with property object suggests that once docstrings are supported, there don't seem to be any other needs. But then, you never can tell ;-) Raymond "Difficult to see. Always in motion is the future." -- Master Yoda
19.03.19 20:55, Raymond Hettinger пише:
I'm working on ways to make improve help() by giving docstrings to member objects.
One way to do it is to wait until after the class definition and then make individual, direct assignments to __doc__ attributes.This way widely the separates docstrings from their initial __slots__ definition. Working downstream from the class definition feels awkward and doesn't look pretty.
There's another way I would like to propose¹. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
I think it would be nice to separate docstrings from the bytecode. This would be allow to have several translated sets of docstrings and load an appropriate set depending on user preferences. This would help in teaching Python. It is possible with docstrings of modules, classes, functions, methods and properties (created by using the decorator), because the compiler knows what string literal is a docstring. But this is impossible with namedtuple fields and any of the above ideas for slots. It would be nice to allow to specify docstrings for slots as for methods and properties. Something like in the following pseudocode: class NormalDist: slot mu: '''Arithmetic mean''' slot sigma: '''Standard deviation''' It would be also nice to annotate slots and add default values (used when the slot value was not set). class NormalDist: mu: float = 0.0 '''Arithmetic mean''' sigma: float = 1.0 '''Standard deviation'''
(answers above and below the quoting)
I like the idea of documenting attributes, but we shouldn't force the user
to use __slots__ as that has significant side effects and is rarely
something people should bother to use. There are multiple types of
attributes. class and instance. but regardless of where they are
initialized, they all define the API shape of a class (or instance).
Q: Where at runtime regardless of syntax chosen would such docstrings
live? One (of many) common conventions today is to just put them into an
Attributes: or similar section of the class docstring. We could actually
do that automatically by appending a section to the class docstring, but
that unstructures the data enshrining one format and could break existing
code for the users of the few but existing APIs that treat docstrings as
structured runtime data instead of documentation if someone were to try and
use attribute docstrings on subclasses of those library types. (ply does
this, I believe some database abstraction APIs do as well).
On Wed, Mar 20, 2019 at 12:41 AM Serhiy Storchaka
I'm working on ways to make improve help() by giving docstrings to member objects.
One way to do it is to wait until after the class definition and then make individual, direct assignments to __doc__ attributes.This way widely
19.03.19 20:55, Raymond Hettinger пише: the separates docstrings from their initial __slots__ definition. Working downstream from the class definition feels awkward and doesn't look pretty.
There's another way I would like to propose¹. The __slots__ definition
already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
I think it would be nice to separate docstrings from the bytecode. This would be allow to have several translated sets of docstrings and load an appropriate set depending on user preferences. This would help in teaching Python.
It is possible with docstrings of modules, classes, functions, methods and properties (created by using the decorator), because the compiler knows what string literal is a docstring. But this is impossible with namedtuple fields and any of the above ideas for slots.
It would be nice to allow to specify docstrings for slots as for methods and properties. Something like in the following pseudocode:
class NormalDist: slot mu: '''Arithmetic mean''' slot sigma: '''Standard deviation'''
I don't think adding a 'slot' keyword even if limited in scope to class body definition level is a good idea (very painful anytime we reserve a new word that is already used in code and APIs).
It would be also nice to annotate slots and add default values (used when the slot value was not set).
class NormalDist: mu: float = 0.0 '''Arithmetic mean''' sigma: float = 1.0 '''Standard deviation'''
Something along these lines is more interesting to me. And could be applied to variables in _any_ scope. though there wouldn't be a point in using a string in context where the name isn't bound to a class or module. The best practice today remains "just use the class docstring to document your public class and instance attributes". FWIW other languages tend to generate their documentation from code via comments rather than requiring a special in language runtime accessible syntax to declare it as documentation. It feels like Python is diverging from the norm if we were encourage more of this __doc__ carried around at runtime implicit assignment than we already have. I'm not convinced that is a good thing. -gps
On Mar 20, 2019, at 3:30 PM, Gregory P. Smith
wrote: I like the idea of documenting attributes, but we shouldn't force the user to use __slots__ as that has significant side effects and is rarely something people should bother to use.
Member objects are like property objects in that they exist at the class level and show up in the help whether you want them to or not. AFAICT, they are the only such objects to not have a way to attach docstrings. For instance level attributes created by __init__, the usual way to document them is in either the class docstring or the __init__ docstring. This is because they don't actually exist until __init__ is run. No one is forcing anyone to use slots. I'm just proposing that for classes that do use them that there is currently no way to annotate them like we do for property objects (which people aren't being forced to use either). The goal is to make help() better for whatever people are currently doing. That shouldn't be controversial. Someone not liking or recommending slots is quite different from not wanting them documented. In the examples I posted (taken from the standard library), the help() is clearly better with the annotations than without. Raymond
On 03/19/2019 11:55 AM, Raymond Hettinger wrote:
I'm working on ways to make improve help() by giving docstrings to member objects.
Cool!
There's another way I would like to propose. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
[...]
What do you all think about the proposal?
This proposal only works with objects defining __slots__, and only the objects in __slots__? Does it help Enum, dataclasses, or other enhanced classes/objects? -- ~Ethan~
On 03/20/2019 03:24 PM, Ethan Furman wrote:
On 03/19/2019 11:55 AM, Raymond Hettinger wrote:
There's another way I would like to propose. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
[...]
What do you all think about the proposal?
This proposal only works with objects defining __slots__, and only the objects in __slots__? Does it help Enum, dataclasses, or other enhanced classes/objects?
Hmm. Said somewhat less snarkily, is there a more general solution to the problem of absent docstrings or do we have to attack this problem piece-by-piece? -- ~Ethan~
On Mar 20, 2019, at 3:59 PM, Ethan Furman
wrote: Hmm. Said somewhat less snarkily, is there a more general solution to the problem of absent docstrings or do we have to attack this problem piece-by-piece?
I think this is the last piece. The pydoc help() utility already knows how to find docstrings for other class level descriptors: property, class method, staticmethod. Enum() already has nice looking help() output because the class variables are assigned values that have a nice __repr__, making them self documenting. By design, dataclasses aren't special -- they just make regular classes, similar to or better than you would write by hand. Raymond
On 19.03.2019 21:55, Raymond Hettinger wrote:
I'm working on ways to make improve help() by giving docstrings to member objects.
One way to do it is to wait until after the class definition and then make individual, direct assignments to __doc__ attributes.This way widely the separates docstrings from their initial __slots__ definition. Working downstream from the class definition feels awkward and doesn't look pretty.
There's another way I would like to propose¹. The __slots__ definition already works with any iterable including a dictionary (the dict values are ignored), so we could use the values for the docstrings.
This keeps all the relevant information in one place (much like we already do with property() objects). This way already works, we just need a few lines in pydoc to check to see if a dict if present. This way also looks pretty and doesn't feel awkward.
I've included worked out examples below. What do you all think about the proposal?
Raymond
¹ https://bugs.python.org/issue36326
====== Desired help() output ======
help(NormalDist) Help on class NormalDist in module __main__:
class NormalDist(builtins.object) | NormalDist(mu=0.0, sigma=1.0) | | Normal distribution of a random variable | | Methods defined here: | | __init__(self, mu=0.0, sigma=1.0) | NormalDist where mu is the mean and sigma is the standard deviation. | | cdf(self, x) | Cumulative distribution function. P(X <= x) | | pdf(self, x) | Probability density function. P(x <= X < x+dx) / dx | | ---------------------------------------------------------------------- | Data descriptors defined here: | | mu | Arithmetic mean. | | sigma | Standard deviation. | | variance | Square of the standard deviation.
====== Example of assigning docstrings after the class definition ======
class NormalDist: 'Normal distribution of a random variable'
__slots__ = ('mu', 'sigma')
def __init__(self, mu=0.0, sigma=1.0): 'NormalDist where mu is the mean and sigma is the standard deviation.' self.mu = mu self.sigma = sigma
@property def variance(self): 'Square of the standard deviation.' return self.sigma ** 2.
def pdf(self, x): 'Probability density function. P(x <= X < x+dx) / dx' variance = self.variance return exp((x - self.mu)**2.0 / (-2.0*variance)) / sqrt(tau * variance)
def cdf(self, x): 'Cumulative distribution function. P(X <= x)' return 0.5 * (1.0 + erf((x - self.mu) / (self.sigma * sqrt(2.0))))
NormalDist.mu.__doc__ = 'Arithmetic mean' NormalDist.sigma.__doc__ = 'Standard deviation'
IMO this is another manifestation of the problem that things in the class definition have no access to the class object. Logically speaking, a definition item should be able to see everything that is defined before it. For the same reason, we have to jump through hoops to use a class name in a class attribute definition -- see e.g. https://stackoverflow.com/questions/14513019/python-get-class-name If that problem is resolved, you would be able to write something like: class NormalDist: 'Normal distribution of a random variable' __slots__ = ('mu', 'sigma') __self__.mu.__doc__= 'Arithmetic mean' __self__.sigma.__doc__= 'Stndard deviation'
====== Example of assigning docstrings with a dict =====
class NormalDist: 'Normal distribution of a random variable'
__slots__ = {'mu' : 'Arithmetic mean.', 'sigma': 'Standard deviation.'}
def __init__(self, mu=0.0, sigma=1.0): 'NormalDist where mu is the mean and sigma is the standard deviation.' self.mu = mu self.sigma = sigma
@property def variance(self): 'Square of the standard deviation.' return self.sigma ** 2.
def pdf(self, x): 'Probability density function. P(x <= X < x+dx) / dx' variance = self.variance return exp((x - self.mu)**2.0 / (-2.0*variance)) / sqrt(tau * variance)
def cdf(self, x): 'Cumulative distribution function. P(X <= x)' return 0.5 * (1.0 + erf((x - self.mu) / (self.sigma * sqrt(2.0))))
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru
-- Regards, Ivan
On Mar 20, 2019, at 3:47 PM, Ivan Pozdeev via Python-Dev
wrote: NormalDist.mu.__doc__ = 'Arithmetic mean' NormalDist.sigma.__doc__ = 'Standard deviation'
IMO this is another manifestation of the problem that things in the class definition have no access to the class object. Logically speaking, a definition item should be able to see everything that is defined before it.
The member objects get created downstream by the type() metaclass. So, there isn't a visibility issue because the objects don't exist yet. Raymond
participants (7)
-
Abdur-Rahmaan Janhangeer
-
Ethan Furman
-
Gregory P. Smith
-
Ivan Pozdeev
-
MRAB
-
Raymond Hettinger
-
Serhiy Storchaka