Descriptor write-up [Draft: Please comment]
In the spirit of Michele Simionato's write-up on method resolution order, I've written up almost everything I know about descriptors: http://tinyurl.com/d63d Key features: * Defines a descriptor * Shows the protocol and how it is called * Shows a handmade descriptor * Then covers functions, properties, static and class methods * Gives examples of each * Suggests when each would be useful * Gives a pure python definition of each * Provides exact references into C source to save efforts in hunting down details I would like to hammer this into something really useful. So, any and all suggestions are welcome. Raymond Hettinger P.S. I've borrowed Alex Martelli's style of making precise descriptions of how the CPython works by showing pure Python equivalents.
Raymond Hettinger wrote:
I would like to hammer this into something really useful. So, any and all suggestions are welcome.
To be really useful, it should be part of the standard Python documentation. So you should find a strategy to move this to the three major components (tutorial, language ref, library ref). I think some of it should go into section 3 of the language reference, and the rest into the library reference (although there is already documentation for property and classmethod - perhaps some of the builtins deserve their own subsections). Perhaps the tutorial should be extended to include examples for the "typical use cases". Regards, Martin
At 04:32 AM 6/1/03 -0400, Raymond Hettinger wrote:
In the spirit of Michele Simionato's write-up on method resolution order, I've written up almost everything I know about descriptors: http://tinyurl.com/d63d
Key features: * Defines a descriptor * Shows the protocol and how it is called * Shows a handmade descriptor * Then covers functions, properties, static and class methods * Gives examples of each * Suggests when each would be useful * Gives a pure python definition of each * Provides exact references into C source to save efforts in hunting down details
I would like to hammer this into something really useful. So, any and all suggestions are welcome.
It would be a good idea to add some information about "data" and "non-data" descriptors, and the differences of how their attribute lookup is processed. I recently posted here about the "attribute lookup process" or something to that effect, which covered this. Understanding data vs. non-data descriptors is important if you want to do pretty much anything with descriptors beyond what property() does. This section: "Alternatively, a descriptor is invoked automatically upon attribute access. For example, obj.a looks up a in the dictionary of obj. If obj defines the method __get__, then it is automatically invoked if obj is a new style class or object." isn't accurate. I think you meant to say that if 'a' defines __get__, then it's invoked. But this isn't accurate either. If obj.__dict__ has an 'a' entry, and the descriptor is a non-data descriptor, __get__ will be ignored and the contents of obj.__dict__ will be returned. (One interesting effect of this is that you can create a non-data descriptor with a __get__ that performs a computation and stashes the result in the instance dictionary - from then on the computed value is returned.) The Python code you have for the algorithm is also incorrect. Here's a more accurate depiction of the process. It's not a straight translation of PyGeneric_GetAttr, but an attempt to do as you have done, i.e. write a pure Python __getattribute__ with equivalent semantics. def __getattribute__(self, key): dict = object.__getattribute__(self, '__dict__') try: # Does the class have a descriptor for this attribute? descr = getattr(type(self),key) except AttributeError: # No, just use what's in the dictionary try: return dict[key] except KeyError: raise AttributeError if hasattr(descr,'__set__') and hasattr(descr,'__get__'): # It's a data descriptor, so use the get method return descr.__get__(self,type(self)) # non-data descriptor, use __dict__ first, *then* __get__ try: return dict[key] except KeyError: if hasattr(descr, '__get__'): return descr.__get__(self, type(self)) # not a descriptor, return it as-is return descr As you can see, it's a bit more complex than your writeup implies. :)
(I'd like to review this more carefully when I have more time, but one thing jumped out at me:) [Phillip Eby]
It would be a good idea to add some information about "data" and "non-data" descriptors, and the differences of how their attribute lookup is processed. I recently posted here about the "attribute lookup process" or something to that effect, which covered this. Understanding data vs. non-data descriptors is important if you want to do pretty much anything with descriptors beyond what property() does.
Note that I recently changed the treatment of data descriptors by super() in 2.3. typeobject.c rev 2.227/2.228: ---------------------------- revision 2.228 date: 2003/04/16 20:01:36; author: gvanrossum; state: Exp; lines: +1 -1 Sigh. The crucial change was still missing from the previous checkin. :-( ---------------------------- revision 2.227 date: 2003/04/16 19:40:58; author: gvanrossum; state: Exp; lines: +10 -4 - super() no longer ignores data descriptors, except __class__. See the thread started at http://mail.python.org/pipermail/python-dev/2003-April/034338.html ---------------------------- --Guido van Rossum (home page: http://www.python.org/~guido/)
"Raymond Hettinger"
I would like to hammer this into something really useful. So, any and all suggestions are welcome.
Something about data descriptors (ones defining __set__) and method descriptors (ones not defining __set__) and their differences would be cool (particularly as this is something I don't quite understand!). Cheers, M. -- Those who have deviant punctuation desires should take care of their own perverted needs. -- Erik Naggum, comp.lang.lisp
Something about data descriptors (ones defining __set__) and method descriptors (ones not defining __set__) and their differences would be cool (particularly as this is something I don't quite understand!).
The main difference is that a non-data descriptor (they're not just for methods) can be overridden by something in the instance __dict__, while a data descriptor *always* overrides the instance __dict__. This is done mostly for backwards compatibility, although it is useful enough to call it a feature: you can define a method, and then assign some other function (or callable) to the corresponding instance variable, and this will hide the method *for that instance*. (Sort of like treating the instance as a subclass of the class, but not quite. :-) Data descriptors exist so that you can also have a different behavior, which is also useful: setting the attribute calls the data descriptor's __set__ method and doesn't affect the instance __dict__. This is so that you can implement properties without overriding __setattr__ (the horrible overkill hack that had to be used before 2.2 to trap setting even a single attribute). And note that to create a read-only attribute, you create a data descriptor whose __set__ raises an exception. (This explains why a property which has only a 'get' function still has the __set__ attribute.) --Guido van Rossum (home page: http://www.python.org/~guido/)
On Sun, Jun 01, 2003, Raymond Hettinger wrote:
In the spirit of Michele Simionato's write-up on method resolution order, I've written up almost everything I know about descriptors:
Cool! Haven't had a chance to look at it, but I'd suggest sticking with a canonical URL since it isn't very long: http://users.rcn.com/python/download/Descriptor.htm -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ "In many ways, it's a dull language, borrowing solid old concepts from many other languages & styles: boring syntax, unsurprising semantics, few automatic coercions, etc etc. But that's one of the things I like about it." --Tim Peters on Python, 16 Sep 93
participants (6)
-
"Martin v. Löwis" -
Aahz -
Guido van Rossum -
Michael Hudson -
Phillip J. Eby -
Raymond Hettinger