[issue42781] functools.cached_property docs should explain that it is non-overriding
New submission from Luciano Ramalho <luciano@ramalho.org>: functools.cached_property is a great addition to the standard library, thanks! However, the docs do not say that @cached_property produces a non-overriding descriptor, in contrast with @property. If a user replaces a @property with a @cached_property, her code may or may not break depending on the existence of an instance attribute with the same name as the decorated method. This is surprising and may affect correctness, so it deserves even more attention than the possible performance loss already mentioned in the docs, related to the shared-dict optimization. In the future, perhaps we can add an argument to @cached_property to optionally make it produce overriding descriptors. ---------- assignee: docs@python components: Documentation messages: 384019 nosy: docs@python, ramalho priority: normal severity: normal status: open title: functools.cached_property docs should explain that it is non-overriding versions: Python 3.10, Python 3.8, Python 3.9 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment: FYI, the usual term is "data descriptor" instead of "overriding descriptor". Normally the docs for things like property() and cached_property() don't mention the term descriptor at all. From the user point of view, that is an implementation detail. What is important is what it does and how to use it. In the case of cached_property(), the docs do a pretty good job, "Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance."
From a user point of view, that is exactly what happens. The method is called exactly once. The result is cached in an attribute and it behaves like a normal attribute for the remaining life (effectively, the descriptor is gone.
I would suggest a small amendment, and say "cached as a normal attribute with the same name". The choice of attribute isn't arbitrary, it is exactly the same as the cached property.
In the future, perhaps we can add an argument to @cached_property to optionally make it produce overriding descriptors.
That doesn't make any sense to me. It would defeat the entire mechanism for cached_property().
If a user replaces a @property with a @cached_property, her code may or may not break depending on the existence of an instance attribute with the same name as the decorated method.
We've never had a report of an issue like this and I don't expect to see over. In general, if someone is using property(), it is already a bit challenging to store and retrieve attributes that have the same name as the property. I think it is sufficient to just state that cached_property() uses the attribute with the same name as the property. That way, it won't be a "surprise" that attribute with the same name gets used :-) One other possible addition is to note that the attribute is writeable: class A: @cached_property def x(self): print('!') return 42
a = A() vars(a) # Initially, there is no instance variable {} a.x # Method is called ! 42 vars(a) # Data is stored in the instance variable {'x': 42} a.x # Method is not called again 42 a.x = 10 # Attribue is writeable vars(a) {'x': 10} a.x 10
---------- nosy: +rhettinger _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Luciano Ramalho <luciano@ramalho.org> added the comment:
FYI, the usual term is "data descriptor" instead of "overriding descriptor".
Yes, the Python docs are consistent in always using "data descriptor", but I've adopted "overriding descriptor" from Martelli's Python in a Nutshell--in Fluent Python I also put a note saying that "data descriptor" is used in the docs. I think "overriding descriptor" is more descriptive. In particular, I find "non-data descriptor" quite puzzling. But this issue is not about changing the jargon.
Normally the docs for things like property() and cached_property() don't mention the term descriptor at all. From the user point of view, that is an implementation detail.
I'd agree, if I wasn't personally bitten by the issue I reported. I was refactoring an existing class which had hand-made caches in a couple of methods decorated with @property. When I discovered @cached_property, I tried to simplify my code by using it, and it broke my code in one place, but not in the other. Leonardo Rochael had experience with cached_property and told me about the difference. I suggest a warning noting the different behavior regarding existing instance attributes. The warning doesn't need to assume the user knows what is a descriptor, but it should in my opinion point to your excellent Descriptor HowTo Guide for more information.
I would suggest a small amendment, and say "cached as a normal attribute with the same name". The choice of attribute isn't arbitrary, it is exactly the same as the cached property.
That's good too.
In the future, perhaps we can add an argument to @cached_property to optionally make it produce overriding descriptors.
That doesn't make any sense to me. It would defeat the entire mechanism for cached_property().
My idea would be to add a dummy __set__ method if the overriding option was True (default would be False). The method could raise an exception. But its presence would make @cached_property behave more like @property in the most common use case of @property: as an overriding descriptor emulating a read-only attribute.
If a user replaces a @property with a @cached_property, her code may or may not break depending on the existence of an instance attribute with the same name as the decorated method.
We've never had a report of an issue like this and I don't expect to see over.
You just did ;-). I filed this issue after spending hours trying to figure out what the problem was in my code on my second attempt at using @cached_property. I expected @cached_property would work as a drop-in replacement for @property when emulating a read-only attribute, and it did not.
One other possible addition is to note that the attribute is writeable:
Yes, the code snippet you suggested is a good way of saying "this produces a non-overriding descriptor". However we want to say it, I think it is important to contrast the behavior of @cached_property v. @property regarding the presence of attributes with the same name. Cheers and a happy 2021 with two doses of vaccine for you and your loved ones, Raymond! ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Change by Raymond Hettinger <raymond.hettinger@gmail.com>: ---------- nosy: -rhettinger _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Change by Raymond Hettinger <raymond.hettinger@gmail.com>: ---------- keywords: +patch nosy: +rhettinger nosy_count: 2.0 -> 3.0 pull_requests: +22872 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24031 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment: New changeset c8a7b8fa1b5f9a6d3052db792018dc27c5a3a794 by Raymond Hettinger in branch 'master': bpo-42781: Document the mechanics of cached_property from a user viewpoint (GH-24031) https://github.com/python/cpython/commit/c8a7b8fa1b5f9a6d3052db792018dc27c5a... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- nosy: +miss-islington nosy_count: 3.0 -> 4.0 pull_requests: +22876 pull_request: https://github.com/python/cpython/pull/24035 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Change by Raymond Hettinger <raymond.hettinger@gmail.com>: ---------- assignee: docs@python -> rhettinger resolution: -> fixed stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment: New changeset 8333d421c0d7b90de3ff92002af9fd2c5d5f373c by Miss Islington (bot) in branch '3.9': bpo-42781: Document the mechanics of cached_property from a user viewpoint (GH-24031) (#24035) https://github.com/python/cpython/commit/8333d421c0d7b90de3ff92002af9fd2c5d5... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue42781> _______________________________________
participants (3)
-
Luciano Ramalho -
miss-islington
-
Raymond Hettinger