Hot-To Guide for Descriptors - Nits! :-)

Hello, Raymond. Cc:ing python-dev, so others can bash me as well! :-) This is about `http://users.rcn.com/python/download/Descriptor.htm'. Before anything else, let me thank you for having written this essay! Please also take all my remarks with a grain of salt or two, as I'm far for pretending I really understand all the matters here. * In `Definition and Introduction', a parenthetical note "(ones that subclass `object' or `type')" qualifies the expression "new style objects or classes". Not being an English speaker, I'm a little mixed up with the plural "ones" and the fact I was expecting "subclasses" instead of "subclass", maybe replacing "ones" by "those" would be clearer. * Another point is that the phrasing suggests that the parenthetical note is a definition for what precedes it. In my understanding, the parenthetical note implies what precedes it, but is not implied by it. A new-style instance may well have no base class at all, only given its meta-class is a subtype of `type'. Would it be more precise to state: "... new style objects or classes (those for which the meta-class is a subtype of `type')"? Being sub-classed from object or type is just a way, among others, for identifying `type' as the meta-class; but being sub-classed from object is not really required. * In `Descriptor Protocol', it might help a bit saying immediately a word or two about the `type=None' argument. Looking down for some hint, the box holding "def __getattribute__(self, key): ..." uses the statement "return v.__get__(None, self)", which I find confusing respective to the `__get__' prototype, as `obj' corresponds to None, and `type=None' likely corresponds to an instance instead of a type. * In `Invoking Descriptors', it might help a bit insisting on the (presumed) fact that `hasattr' itself does not look for descriptors. For symmetry, it would also be nice having two boxes defining `__getattribute__, one for objects, another for classes, instead of only the second one. I was not sure if the assymetry had a purpose, like maybe telling us that the first case may not be expressed using Python. * A little below is a quick description of `super()'. It says that "The call ... searches ... mro ... for the base class ... immediately preceding ...". Should not it be `following' instead of `preceding'? In case yes, than maybe `A' and `B' might be exchanged for clarity. * In `Properties', the box starting with "class Property(object):" raises a few questions. First is the `doc or ""' to defined `self.__doc__', it does not seem that the ` or ""' exists in Python 2.2.1 nor 2.3.2, the __doc__ really defaults to None. Second is that the argument `objtype=None' to `__get__' is unused, and once more, this does not help understanding its purpose, maybe some explanation would be welcome about why it exists and is unneeded in the context of `property'. Third is that on the last line, "self.fdel(obj, value)" was probably meant to be "self.fdel(obj)". -- François Pinard http://www.iro.umontreal.ca/~pinard

* In `Definition and Introduction', a parenthetical note "(ones that subclass `object' or `type')" qualifies the expression "new style objects or classes". Not being an English speaker, I'm a little mixed up with the plural "ones" and the fact I was expecting "subclasses" instead of "subclass", maybe replacing "ones" by "those" would be clearer.
Clarified.
* Another point is that the phrasing suggests that the parenthetical note is a definition for what precedes it. In my understanding, the parenthetical note implies what precedes it, but is not implied by it.
Sounds like an obscure grammar nit (one that doesn't appear in any of my grammar books). I think it reads well enough as is.
A new-style instance may well have no base class at all, only given its meta-class is a subtype of `type'. Would it be more precise to state: "... new style objects or classes (those for which the meta-class is a subtype of `type')"? Being sub-classed from object or type is just a way, among others, for identifying `type' as the meta-class; but being sub-classed from object is not really required.
Nope, new-style is taken to mean objects/classes inheriting from object/type. Meta-class objects are neither new-style nor old-style. While there is room to argue with this arbitrary distinction, it is consistent with Guido's essay and especially relevant to my article because most of the rules don't necessarily apply when meta-classes are used. This is because the machinery for descriptors is embedded in type.__getattribute__ and object.__getattribute__. Override or fail to inherit either of these and all bets are off.
* In `Descriptor Protocol', it might help a bit saying immediately a word or two about the `type=None' argument. Looking down for some hint, the box holding "def __getattribute__(self, key): ..." uses the statement "return v.__get__(None, self)", which I find confusing respective to the `__get__' prototype, as `obj' corresponds to None, and `type=None' likely corresponds to an instance instead of a type.
I'll keep an open mind on this one but think it is probably fine as it stands. Though the calling pattern may not be beautiful and could be confusing to some, my description accurately represents the signature and cases where are argument is potentially ignored. That is just the way it is.
* In `Invoking Descriptors', it might help a bit insisting on the (presumed) fact that `hasattr' itself does not look for descriptors.
I considered this and other explanations when writing the article. The subject is already complex enough that additional explanations can make it harder to understand rather than easier. Sidetrips can add more confusion than they cure. I see no reason to delve into what hasattr() doesn't do.
For symmetry, it would also be nice having two boxes defining `__getattribute__, one for objects, another for classes, instead of only the second one. I was not sure if the assymetry had a purpose, like maybe telling us that the first case may not be expressed using Python.
It did have a purpose. I put in pure python code only when it served as an aid to understanding. In this case, the operation of __getattribute__ is more readily described with prose. I do include a C code reference for those wanting all the gory details. If you look at the reference, you'll see that a pure python equivalent would not be succinct or provide any deep insights not covered the text. Once again, it was a question of whether to break-up the flow of the argument/explanation. I like it the way it is.
* A little below is a quick description of `super()'. It says that "The call ... searches ... mro ... for the base class ... immediately preceding ...". Should not it be `following' instead of `preceding'?
It is a matter of perspective. A is defined before its derived class B and can be said to precede it in the inheritance graph; however, the MRO list would have B ... A ... O so there is a case for the word "following". To keep consistent with Guido's essay, I'll make the change and adopt your convention.
In case yes, than maybe `A' and `B' might be exchanged for clarity.
No. A is higher than B on the inheritance hierarchy. Also, the lettering is consistent with Guido's essay.
* In `Properties', the box starting with "class Property(object):" raises a few questions. First is the `doc or ""' to defined `self.__doc__', it does not seem that the ` or ""' exists in Python 2.2.1 nor 2.3.2, the __doc__ really defaults to None.
Fixed.
Second is that the argument `objtype=None' to `__get__' is unused, and once more, this does not help understanding its purpose, maybe some explanation would be welcome about why it exists and is unneeded in the context of `property'.
That's the way it is.
Third is that on the last line, "self.fdel(obj, value)" was probably meant to be "self.fdel(obj)".
Fixed. Raymond Hettinger

[Raymond Hettinger]
Clarified. [...] Fixed. [...] (etc.)
Thanks, Raymond, for all those.
This is because the machinery for descriptors is embedded in type.__getattribute__ and object.__getattribute__. Override or fail to inherit either of these and all bets are off.
I thinks I understand better what you are saying. Maybe part of my problem is that I'm not sure how `__getattribute__' is itself fetched. While I know you do not want to speak about metatypes, it might be that whenever you write `type.__getattribute__', you really mean the `__getattribute__' found in the dict of the metatype (or is it?); while when you write `object.__getattribute__', you really mean the `__getattribute__' found by scanning the base classes of the current class, and `object' always when there is no base class for a "new-style" class. These are sensible hypotheses, yet I'm not sure, of course. Maybe that failing to say the full truth, _if_ this is the case, might raise questions more than calm them! :-)
* In `Descriptor Protocol', it might help a bit saying immediately a word or two about the `type=None' argument. Looking down for some hint, the box holding "def __getattribute__(self, key): ..." uses the statement "return v.__get__(None, self)", which I find confusing respective to the `__get__' prototype, as `obj' corresponds to None, and `type=None' likely corresponds to an instance instead of a type.
I'll keep an open mind on this one but think it is probably fine as it stands. Though the calling pattern may not be beautiful and could be confusing to some, my description accurately represents the signature and cases where are argument is potentially ignored. That is just the way it is.
Just adding your own quote above might help the How-To. About: descr.__get__(self, obj, type=None) --> value I really think it would help readers like me saying that the pattern is not beautiful, that `obj' may be None, and that `type' may be something else than a type, and this is the way it is. It would invite the reader to swallow the snake, instead of fighting it.
* In `Invoking Descriptors', it might help a bit insisting on the (presumed) fact that `hasattr' itself does not look for descriptors.
I see no reason to delve into what hasattr() doesn't do.
One has to memorise a few rules about the order in which the lookups are done for finding an attribute, and unless `hasattr' is more used as pseudo-code than Python code, my fear is that `hasattr' does not look at the same place, nor in the same order, as the few rules above. Maybe there is no danger of any confusion, maybe there is, I surely wondered.
Second is that the argument `objtype=None' to `__get__' is unused, and once more, this does not help understanding its purpose, maybe some explanation would be welcome about why it exists and is unneeded in the context of `property'.
That's the way it is.
I would not doubt it, but my suggestion was that the How-To could offer more help for readers to understand or find some logic in the way it is, enough to get a grasp and assimilate the notion. If there is logic to present, merely saying that it should be assimilated without understanding might help readers like me at not stumbling on the matter for too long. I hope you read my criticism or requests as constructive... -- François Pinard http://www.iro.umontreal.ca/~pinard

François Pinard <pinard@iro.umontreal.ca> wrote:
This is because the machinery for descriptors is embedded in type.__getattribute__ and object.__getattribute__. Override or fail to inherit either of these and all bets are off.
I thinks I understand better what you are saying. Maybe part of my problem is that I'm not sure how `__getattribute__' is itself fetched. While I know you do not want to speak about metatypes, it might be that whenever you write `type.__getattribute__', you really mean the `__getattribute__' found in the dict of the metatype (or is it?); while when you write `object.__getattribute__', you really mean the `__getattribute__' found by scanning the base classes of the current class, and `object' always when there is no base class for a "new-style" class.
As always, a little experiment at the interactive prompt is instructive: Python 2.3.3 (#1, Dec 21 2003, 09:30:26) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
class A : ... __metaclass__ = type ... A <class '__main__.A'> A.__bases__ (<type 'object'>,) class B : ... pass ... B.__bases__ ()
IMHO, it does make sense to present a simplified view in introductory documentation, but I also think that there should be footnotes explaining where white lies are told and giving hints of the full picture with all the attendant complexity. -- Christian Tanzer http://www.c-tanzer.at/

[Christian Tanzer]
François Pinard <pinard@iro.umontreal.ca> wrote:
[...] found by scanning the base classes of the current class, and `object' always when there is no base class for a "new-style" class.
As always, a little experiment at the interactive prompt is instructive: [...]
Indeed. One more tiny experiment is also enlightening: Python 2.3.3 (#1, Jan 21 2004, 22:36:17)
__metaclass__ = type class A: pass ... A.__bases__ (<type 'object'>,) A.__mro__ (<class '__main__.A'>, <type 'object'>)
(I merely added __mro__ to your example) which seems to confirm that `object' is always implied if the metaclass is `type', exactly as if it was explicitly listed as a base. Unless the above experiment just reveals some unspecified behaviour, which just happens to not correctly be in-lined with Guido's intents? That is possible, yet it seems unlikely. Only Guido really knows! :-) Maybe we should not overly state that one ought to explicitly sub-class a class from `object' to get a new style class, nor suggest that not doing so might yield broken classes with unpredictable behaviour. Of course, we have been educated to read "class A(object):" as the standard way to flag `A' as being new-style, but if I could avoid adding such "(object)" everywhere in the code, and merely declare __metaclass__ once per module, it looks much neater to me. Taste varies :-). More it goes, more it looks like my fears were not really sounded. Thanks to all those who participated in clarifying this little issue. -- François Pinard http://www.iro.umontreal.ca/~pinard
participants (3)
-
François Pinard
-
Raymond Hettinger
-
tanzer@swing.co.at