Hi Steve, Here are a couple of code snippets. I hope it's not too verbose :) Setting property on the __doc__ works on the instance, but not on the class: class A: @property def __doc__(self): return "Works only on the instance" a = A() print('A.__doc__ is {}'.format(A.__doc__)) # A.__doc__ is <property object at 0x7fa48af86e58> print('a.__doc__ is {}'.format(a.__doc__)) # a.__doc__ is Works only on the instance help(A) # Help on class A in module __main__: # # class A(builtins.object) # | Data descriptors defined here: # | # | __dict__ # | dictionary for instance variables (if defined) # | # | __weakref__ # | list of weak references to the object (if defined) However, you can fix this with a separate class that follows the property protocol: some_var = "oi" class DocProperty: def __get__(self, instance, owner): return "Dynamic class doc: {}".format(some_var) class A: """Static doc""" __doc__ = DocProperty() print(A.__doc__) # Dynamic class doc: oi some_var = "noi" print(A.__doc__) # Dynamic class doc: noi help(A) # Help on class A in module __main__: # # class A(builtins.object) # | Dynamic class doc: noi # | # | Data descriptors defined here: # | # | __dict__ # | dictionary for instance variables (if defined) # | # | __weakref__ # | list of weak references to the object (if defined) This worked well when I decorate a *class.* Consider now the case where we have to decorate a *function*. I could decorate with a callable class, but __doc__ now does not work and help() becomes misleading (the function signature is lost and the help says "Help on Wrapper"): import functools class DocProperty: def __init__(self, func): self.func = func def __get__(self, instance, owner): return "Func doc is: {!r} and now my part...".format(self.func.__doc__) def decorate(func): class Wrapper: __doc__ = DocProperty(func) def __call__(self, *args, **kwargs): print("Before the call") return func(*args, **kwargs) wrapper = Wrapper() functools.update_wrapper(wrapper=wrapper, wrapped=func) wrapper.__doc__ = DocProperty(func) return wrapper @decorate def some_func(x): """Original doc.""" print("Hi from some_func") print('some_func.__doc__ is {}'.format(some_func.__doc__)) # some_func.__doc__ is <__main__.DocProperty object at 0x7fd134a1d668> help(some_func) # Help on Wrapper in module __main__ object: # # some_func = class Wrapper(builtins.object) # | Func doc is: 'Original doc.' and now my part... # | # | Methods defined here: # | # | __call__(self, *args, **kwargs) # | # | ---------------------------------------------------------------------- # | Data descriptors defined here: # | # | __dict__ # | dictionary for instance variables (if defined) # | # | __weakref__ # | list of weak references to the object (if defined) I tried to decorate the function with a DocProperty, but then hit the wall. Both __doc__ is not "getted" nor does help() works: import functools class DocProperty: def __init__(self, func): self.func = func def __get__(self, instance, owner): return "Func doc is: {!r} and now my part...".format(self.func.__doc__) def decorate(func): def wrapper(*args, **kwargs): print("Before the call") return func(*args, **kwargs) functools.update_wrapper(wrapper=wrapper, wrapped=func) wrapper.__doc__ = DocProperty(func) return wrapper @decorate def some_func(x): """Original doc.""" print("Hi from some_func") some_func(x=3) # Before the call # Hi from some_func print('some_func.__doc__ is {}'.format(some_func.__doc__)) # some_func.__doc__ is <__main__.DocProperty object at 0x7f0551eea438> help(some_func) # Help on function some_func in module __main__: # # some_func(x) I just couldn't figure out how to make the __doc__ attribute of a function a getter. Is this a bug or am I making a mistake? If it's my mistake, is there any other way how to dynamically __doc__ a function or am I forced to set __doc__ of a function at the decoration time? (If you wonder about the use case: I'd like to dynamically generate the docstrings when functions are decorated with contracts from icontract library. Condition functions need to be parsed and re-formatted, so this is something that should be done on-demand, when the user either wants to see the help() or when the sphinx documentation is automatically generated. The docs should not inflict computational overhead during the decoration since normal course of program operation does not need pretty-printed contracts.) Thanks for any pointers! Please let me know if you'd like me to compress one or the other example or explain something in more detail. Cheers, Marko On Sun, 7 Oct 2018 at 11:07, Steven D'Aprano <steve@pearwood.info> wrote:

On Sun, Oct 07, 2018 at 09:25:13AM +0200, Marko Ristin-Kaufmann wrote:

...

Hi, I'm working on decorators that have a dynamic __doc__ property computed at runtime and not at decoration time.

You mean like this?

py> class X: ... @property ... def __doc__(self): ... return "NOBODY expects the %s Inquisition!" % self.nationality ... def __init__(self, nationality): ... self.nationality = nationality ... py> a = X("Spanish") py> b = X("Belgian") py> a.__doc__ 'NOBODY expects the Spanish Inquisition!' py> b.__doc__ 'NOBODY expects the Belgian Inquisition!'

The only downside is that help(a) doesn't do the right thing. I believe that counts as a bug in help().

...

The decorator must return the wrapper as a function and can not return a callable object with __call__ since the "self" argument would not be properly passed through the decorator.

Sorry, I don't get this. Can you demonstrate? Code speaks much louder than words.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Hi Crhis, Have you tried just generating the docstrings at decoration time and

applying them? By the sound of it, they won't change after that, and the only reason to run it later is performance... but have you actually measured a performance hit resulting from that?

I did. On my laptop (Intel i7-4700MQ), it takes about 4-12 milliseconds to parse and reformat a single condition lambda function of a contract. This quickly adds up to a couple of seconds if you have thousands of condition functions to parse. This might not be a problem for a limited code base, but I imagine that it could get inefficient very quickly as soon as the contract usage would spread to the dependencies as well. Not dead-slow inefficient, but inefficient enough to consider whether the automatic doc generation is worth it. Cheers, Marko On Sun, 7 Oct 2018 at 16:46, Chris Angelico <rosuav@gmail.com> wrote:

...

(If you wonder about the use case: I'd like to dynamically generate the docstrings when functions are decorated with contracts from icontract

On Mon, Oct 8, 2018 at 1:41 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote: library. Condition functions need to be parsed and re-formatted, so this is something that should be done on-demand, when the user either wants to see the help() or when the sphinx documentation is automatically generated. The docs should not inflict computational overhead during the decoration since normal course of program operation does not need pretty-printed contracts.)

...

Have you tried just generating the docstrings at decoration time and applying them? By the sound of it, they won't change after that, and the only reason to run it later is performance... but have you actually measured a performance hit resulting from that?

ChrisA _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Mon, Oct 08, 2018 at 01:45:01AM +1100, Chris Angelico wrote:

On Mon, Oct 8, 2018 at 1:41 AM Marko Ristin-Kaufmann <marko.ristin@gmail.com> wrote:

...

(If you wonder about the use case: I'd like to dynamically generate the docstrings when functions are decorated with contracts from icontract library. Condition functions need to be parsed and re-formatted, so this is something that should be done on-demand, when the user either wants to see the help() or when the sphinx documentation is automatically generated. The docs should not inflict computational overhead during the decoration since normal course of program operation does not need pretty-printed contracts.)

Have you tried just generating the docstrings at decoration time and applying them? By the sound of it, they won't change after that, and the only reason to run it later is performance... but have you actually measured a performance hit resulting from that?

That might work for Marko's use-case, but the problem is more general and I'd like to consider the broader picture. Currently instance __doc__ attributes are sometimes ignored by help(). Given these: py> class X: ... pass ... py> a = X() py> b = X() py> a.__doc__ = "Hello" py> b.__doc__ = "Goodbye" I would expect to see the per-instance docstrings, but in 3.6 at least, help(a) and help(b) both give the unhelpful: Help on X in module __main__ object: class X(builtins.object) | Data descriptors defined here: | | __dict__ | dictionary for instance variables (if defined) | | __weakref__ | list of weak references to the object (if defined) (END) As per my earlier post, making the docstrings a property fails from help(). So I think there's some improvement here: - fix help() so it picks up per-instance docstrings, not just the class docstring; - including the case where the docstring is a property. -- Steve

Hi Marko You wrote:

I just couldn't figure out how to make the __doc__ attribute of a function a getter. Is this a bug or am I making a mistake? If it's my mistake, is there any other way how to dynamically __doc__ a function or am I forced to set __doc__ of a function at the decoration time?

Thank you for your clear examples. I'm not sure, but I may have found the problem. As I recall, setting obj.attr to a property won't work. Instead, you have to set type(obj).attr to the property. And now you come up against a problem. You can't set __doc__ on the type 'function', nor can you subclass the type 'function'.

...

...

def fn(): pass type(fn) <class 'function'>

...

...

type(fn).__doc__ 'function(code, globals[, name[, argdefs[, closure]]]) [snip]'

...

...

type(fn).__doc__ = None TypeError: can't set attributes of built-in/extension type 'function'

...

...

class FN(type(fn)): pass TypeError: type 'function' is not an acceptable base type

As I (more vaguely) recall, something was done recently to add a get attribute property that works directly on objects, rather than on instances of a class. But I've not been able to find it. Perhaps I misunderstood PEP 562. Some perhaps relevant URLs: https://www.python.org/dev/peps/pep-0224/ Attribute Docstrings https://www.python.org/dev/peps/pep-0549/ Instance Descriptors https://www.python.org/dev/peps/pep-0562/ Module __getattr__ and __dir__ https://stackoverflow.com/questions/2447353/getattr-on-a-module -- Jonathan

On Sun, Oct 07, 2018 at 06:17:47PM +0200, Marko Ristin-Kaufmann wrote:

Hi, @Jonathan: thanks! I'll have a look at your links.

So there are actually two issues if I understand correctly: * help() ignoring the property getter when invoked on an instance

Not just the property getter, but if you set instance.__doc__ to a string, help ignores that string too.

* built-in class function ignoring the property getter (some_func.__doc__ set to a property returns the property instead of invoking the getter)

Not just functions, it is all instances. Properties are only invoked if they are on the class object, not the instance. py> class X: ... pass ... py> x = X() py> x.spam = property(lambda self: "computed result") py> x.spam <property object at 0xb78b2d9c> py> x.spam.__get__(x) 'computed result' -- Steve