Hello I've already crossed a bunch of articles detailing python's attribute lookup semantic (__dict__, descriptors, order of base class traversing...), but I have never seen, so far, an explanation of WHICH method did waht, exactly. I assumed that getattr(a, b) was the same as a.__getattribute__(b), and that this __getattribute__ method (or the hidden routine replacing it when we don't override it in our class) was in charge of doing the whole job of traversing the object tree, checking descriptors, binding methods, calling __getattr__ on failure etc. However, the test case below shows that __getattribute__ does NOT call __getattr__ on failure. So it seems it's an upper levl machinery, in getattr(), which is in chrge of that last action. Is that on purpose ? Considering that __getattribute__ (at lest, object.__getattribute__) does 90% of the hard job, why are these 10% left ? Can we find somewhere the details of "who must do what" when customizing attribute access ? Shouldn't we inform people about the fact that __getattribute__ isn't sufficient in itself to lookup an attribute ? Thanks for the attention, regards, Pascal ======= INPUT ======= class A(object): def __getattribute__(self, name): print "A getattribute", name return object.__getattribute__(self, name) def __getattr__(self, name): print "A getattr", name return "hello A" class B(A): def __getattribute__(self, name): print "B getattribute", name return A.__getattribute__(self, name) def __getattr__(self, name): print "B getattr", name return "hello B" print A().obj print "---" print B().obj print "---" print getattr(B(), "obj") print "-----" print object.__getattribute__(B(), "obj") # DOES NOT CALL __getattr__() !!! =========== OUTPUT =========== A getattribute obj A getattr obj hello A --- B getattribute obj A getattribute obj B getattr obj hello B --- B getattribute obj A getattribute obj B getattr obj hello B ----- Traceback (most recent call last): File "C:\Users\Pakal\Desktop\test_object_model.py", line 34, in <module> print object.__getattribute__(B(), "obj") # DOES NOT CALL __getattr__() !!!??? AttributeError: 'B' object has no attribute 'obj'
On 19/03/2010 18:58, Pascal Chambon wrote:
Hello
I've already crossed a bunch of articles detailing python's attribute lookup semantic (__dict__, descriptors, order of base class traversing...), but I have never seen, so far, an explanation of WHICH method did waht, exactly.
I assumed that getattr(a, b) was the same as a.__getattribute__(b), and that this __getattribute__ method (or the hidden routine replacing it when we don't override it in our class) was in charge of doing the whole job of traversing the object tree, checking descriptors, binding methods, calling __getattr__ on failure etc.
However, the test case below shows that __getattribute__ does NOT call __getattr__ on failure. So it seems it's an upper levl machinery, in getattr(), which is in chrge of that last action.
Python 3 has the behavior you are asking for. It would be a backwards incompatible change to do it in Python 2 as __getattribute__ *not* calling __getattr__ is the documented behaviour. Python 3.2a0 (py3k:78770, Mar 7 2010, 20:32:50) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
class x: ... def __getattribute__(s, name): ... print ('__getattribute__', name) ... raise AttributeError ... def __getattr__(s, name): ... print ('__getattr__', name) ... a = x() a.b __getattribute__ b __getattr__ b
This list is not really an appropriate place to ask questions like this though, comp.lang.python would be better. All the best, Michael Fooord
Is that on purpose ? Considering that __getattribute__ (at lest, object.__getattribute__) does 90% of the hard job, why are these 10% left ? Can we find somewhere the details of "who must do what" when customizing attribute access ? Shouldn't we inform people about the fact that __getattribute__ isn't sufficient in itself to lookup an attribute ?
Thanks for the attention, regards, Pascal
======= INPUT =======
class A(object):
def __getattribute__(self, name): print "A getattribute", name return object.__getattribute__(self, name)
def __getattr__(self, name): print "A getattr", name return "hello A"
class B(A):
def __getattribute__(self, name): print "B getattribute", name return A.__getattribute__(self, name)
def __getattr__(self, name): print "B getattr", name return "hello B"
print A().obj print "---" print B().obj print "---" print getattr(B(), "obj") print "-----" print object.__getattribute__(B(), "obj") # DOES NOT CALL __getattr__() !!!
=========== OUTPUT ===========
A getattribute obj A getattr obj hello A --- B getattribute obj A getattribute obj B getattr obj hello B --- B getattribute obj A getattribute obj B getattr obj hello B ----- Traceback (most recent call last): File "C:\Users\Pakal\Desktop\test_object_model.py", line 34, in <module> print object.__getattribute__(B(), "obj") # DOES NOT CALL __getattr__() !!!??? AttributeError: 'B' object has no attribute 'obj' _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
Michael Foord a écrit :
On 19/03/2010 18:58, Pascal Chambon wrote:
Hello
I've already crossed a bunch of articles detailing python's attribute lookup semantic (__dict__, descriptors, order of base class traversing...), but I have never seen, so far, an explanation of WHICH method did waht, exactly.
I assumed that getattr(a, b) was the same as a.__getattribute__(b), and that this __getattribute__ method (or the hidden routine replacing it when we don't override it in our class) was in charge of doing the whole job of traversing the object tree, checking descriptors, binding methods, calling __getattr__ on failure etc.
However, the test case below shows that __getattribute__ does NOT call __getattr__ on failure. So it seems it's an upper levl machinery, in getattr(), which is in chrge of that last action.
Python 3 has the behavior you are asking for. It would be a backwards incompatible change to do it in Python 2 as __getattribute__ *not* calling __getattr__ is the documented behaviour.
Python 3.2a0 (py3k:78770, Mar 7 2010, 20:32:50) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
class x: ... def __getattribute__(s, name): ... print ('__getattribute__', name) ... raise AttributeError ... def __getattr__(s, name): ... print ('__getattr__', name) ... a = x() a.b __getattribute__ b __getattr__ b
I'm confused there, because the script you gave behaves the same in python 2.6. And according to the doc, it's normal, getattr() reacts to an AttributeError from __getattribute__, by calling __getattr__ : """ Python 2.6.5 documentation object.__getattribute__(/self/, /name/) Called unconditionally to implement attribute accesses for instances of the class. If the class also defines __getattr__() <http://docs.python.org/reference/datamodel.html#object.__getattr__>, the latter will not be called unless __getattribute__() <http://docs.python.org/reference/datamodel.html#object.__getattribute__> either calls it explicitly or raises an AttributeError <http://docs.python.org/library/exceptions.html#exceptions.AttributeError>. This method should return the (computed) attribute value or raise an AttributeError <http://docs.python.org/library/exceptions.html#exceptions.AttributeError> exception. In order to avoid infinite recursion in this method, its implementation should always call the base class method with the same name to access any attributes it needs, for example, object.__getattribute__(self, name). """ But the point which for me is still unclear, is : does the default implementation of __getattribute__ (the one of "object") call __getattr__ by himself, or does it rely on its caller for that, by raising an AttributeError ? For Python2, it's blatantly the latter case which is favoured, but since it looks like an implementation detail at the moment, I propose we settle it (and document it) once for all.
This list is not really an appropriate place to ask questions like this though, comp.lang.python would be better.
All the best,
Michael Fooord
Sorry if I misposted, I just (wrongly ?) assumed that it was more an undecided, implementation-specific point (since the doc gave possible behaviours for __getattribute__, without precising which one was the default one), and thus targetted the hands-in-core-code audience only. Regards, Pascal
On 20/03/2010 12:00, Pascal Chambon wrote:
Michael Foord a écrit :
On 19/03/2010 18:58, Pascal Chambon wrote:
Hello
I've already crossed a bunch of articles detailing python's attribute lookup semantic (__dict__, descriptors, order of base class traversing...), but I have never seen, so far, an explanation of WHICH method did waht, exactly.
I assumed that getattr(a, b) was the same as a.__getattribute__(b), and that this __getattribute__ method (or the hidden routine replacing it when we don't override it in our class) was in charge of doing the whole job of traversing the object tree, checking descriptors, binding methods, calling __getattr__ on failure etc.
However, the test case below shows that __getattribute__ does NOT call __getattr__ on failure. So it seems it's an upper levl machinery, in getattr(), which is in chrge of that last action.
Python 3 has the behavior you are asking for. It would be a backwards incompatible change to do it in Python 2 as __getattribute__ *not* calling __getattr__ is the documented behaviour.
Python 3.2a0 (py3k:78770, Mar 7 2010, 20:32:50) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin
class x: ... def __getattribute__(s, name): ... print ('__getattribute__', name) ... raise AttributeError ... def __getattr__(s, name): ... print ('__getattr__', name) ... a = x() a.b __getattribute__ b __getattr__ b
I'm confused there, because the script you gave behaves the same in python 2.6. And according to the doc, it's normal, getattr() reacts to an AttributeError from __getattribute__, by calling __getattr__ :
""" Python 2.6.5 documentation
object.__getattribute__(/self/, /name/)
Called unconditionally to implement attribute accesses for instances of the class. If the class also defines __getattr__() <http://docs.python.org/reference/datamodel.html#object.__getattr__>, the latter will not be called unless __getattribute__() <http://docs.python.org/reference/datamodel.html#object.__getattribute__> either calls it explicitly or raises an AttributeError <http://docs.python.org/library/exceptions.html#exceptions.AttributeError>. This method should return the (computed) attribute value or raise an AttributeError <http://docs.python.org/library/exceptions.html#exceptions.AttributeError> exception. In order to avoid infinite recursion in this method, its implementation should always call the base class method with the same name to access any attributes it needs, for example, object.__getattribute__(self, name).
"""
But the point which for me is still unclear, is : does the default implementation of __getattribute__ (the one of "object") call __getattr__ by himself, or does it rely on its caller for that, by raising an AttributeError ? For Python2, it's blatantly the latter case which is favoured, but since it looks like an implementation detail at the moment, I propose we settle it (and document it) once for all.
Ah right, my apologies. So it is still documented behaviour - __getattr__ is obviously called by the Python runtime and not by __getattribute__. (It isn't just by getattr as the same behaviour is shown when doing a normal attribute lookup and not via the getattr function.)
This list is not really an appropriate place to ask questions like this though, comp.lang.python would be better.
All the best,
Michael Fooord
Sorry if I misposted, I just (wrongly ?) assumed that it was more an undecided, implementation-specific point (since the doc gave possible behaviours for __getattribute__, without precising which one was the default one), and thus targetted the hands-in-core-code audience only.
Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't). Michael Foord
Regards, Pascal
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.
Michael Foord wrote:
Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't).
And as for why not: because __getattribute__ implementations need to be able to call object.__getattribute__ without triggering the fallback behaviour. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia ---------------------------------------------------------------
Michael Foord wrote:
Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't).
If __getattribute__ raises an exception, it won't get a chance to do anything else, so something outside of __getattribute__ must catch the AttributeError and calling __getattr__. So I think the docs *are* specifying the behaviour here, if only by implication. -- Greg
Michael Foord a écrit :
On 20/03/2010 12:00, Pascal Chambon wrote:
But the point which for me is still unclear, is : does the default implementation of __getattribute__ (the one of "object") call __getattr__ by himself, or does it rely on its caller for that, by raising an AttributeError ? For Python2, it's blatantly the latter case which is favoured, but since it looks like an implementation detail at the moment, I propose we settle it (and document it) once for all.
Ah right, my apologies. So it is still documented behaviour - __getattr__ is obviously called by the Python runtime and not by __getattribute__. (It isn't just by getattr as the same behaviour is shown when doing a normal attribute lookup and not via the getattr function.)
I really don't see the docs you're referring to ; until I tested myself, I think I had no obvious reasons to guess that __getattribute__ relied on the upper level caller instead of finishing the hard job himself. Nick Coghlan a écrit :
Michael Foord wrote:
Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't).
And as for why not: because __getattribute__ implementations need to be able to call object.__getattribute__ without triggering the fallback behaviour.
Cheers, Nick.
I guess there are cases in which it is beneficial indeed.
Michael Foord wrote:
Well, the documentation you pointed to specifies that __getattr__ will be called if __getattribute__ raises an AttributeError, it just doesn't specify that it is done by object.__getattribute__ (which it isn't).
If __getattribute__ raises an exception, it won't get a chance to do anything else, so something outside of __getattribute__ must catch the AttributeError and calling __getattr__. So I think the docs *are* specifying the behaviour here, if only by implication.
I don't follow you there - in my mind, the default __getattribute__ could simply have wrapped all its operations inside soem kind of "try..catch AttributeError:" mechanism, and thus been able to fallback to __getattr__ in any way. If I sum it up properly the semantic is : -A.obj and getattr(A, "obj") are exactly the same -They trigger the calling of __getattribute__ on the object (or it's python core equivalent) -By default, this __getattribute__ browse the whole object hierarchy according to well known rules (__dict__, type, type's ancestors..), handling descriptor protocols and the like. But it doesn't fallback to __getattr__ - it raises an AttributeError instead. -getattr() falls back to __getattr__ if __getattribute__ fails -customized __getattribute__ methods have the choice between calling __getattr__ by themselves, or delegating it to getattr() by raising an exception. Wouldn't it be worth completing the doc with these point ? They really didn't seem obvious to me basically (even though, after analysis, some behaviours make more sense than others). I might submit a patch. regards, Pascal
Pascal Chambon wrote:
I don't follow you there - in my mind, the default __getattribute__ could simply have wrapped all its operations inside soem kind of "try..catch AttributeError:" mechanism, and thus been able to fallback to __getattr__ in any way.
But then it would be incorrect to say that "__getattribute__ raises an exception". When we say that a function raises an exception, we normally mean that the exception propagates out of the function and can be seen by the caller, not that it was raised and caught somewhere inside the function. -- Greg
Greg Ewing a écrit :
Pascal Chambon wrote:
I don't follow you there - in my mind, the default __getattribute__ could simply have wrapped all its operations inside soem kind of "try..catch AttributeError:" mechanism, and thus been able to fallback to __getattr__ in any way.
But then it would be incorrect to say that "__getattribute__ raises an exception".
When we say that a function raises an exception, we normally mean that the exception propagates out of the function and can be seen by the caller, not that it was raised and caught somewhere inside the function.
Indeed, but I've never run into any doc mentionning that the default __getattribute__ raised in exception instead of forwarding to __getattr__ by itself. All I've found is "If the class also defines __getattr__() <http://docs.python.org/reference/datamodel.html#object.__getattr__>, the latter will not be called unless __getattribute__() <http://docs.python.org/reference/datamodel.html#object.__getattribute__> either calls it explicitly or raises an AttributeError <http://docs.python.org/library/exceptions.html#exceptions.AttributeError>"; that sentence which simply offers two alternatives for the behaviour of customized __gettattribute__ methods, without giving any hint on the behaviourthat was chosen when implementing object.__gettattribute__. Or am I missing some other doc which I'm supposed to know :? "In the face of ambiguity, refuse the temptation to guess", as we say anyway, so I propose we patch the doc to clarify this point for newcomers ^^ Regards, Pascal
Pascal Chambon wrote:
All I've found is "If the class also defines __getattr__(), the latter will not be called unless __getattribute__() either calls it explicitly or raises an AttributeError
Hmmm. Well, it still implies that there is some mechanism outside of __getattribute__ that will catch an AttributeError and call __getattr__ for you. Given the existence of that mechanism, it seems unlikely that the same thing would be implemented over again in the standard __getattribute__ method. So I don't think it requires very much guesswork to infer that the standard __getattribute__ won't call __getattr__ on its own. I concede that the wording could be improved to remove any possibility of doubt, though. -- Greg
participants (4)
-
Greg Ewing -
Michael Foord -
Nick Coghlan -
Pascal Chambon