[ python-Bugs-643841 ] New class special method lookup change

[SourceForge.net]

Bugs item #643841, was opened at 2002-11-26 09:37
Message generated for change (Comment added) made by ncoghlan
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643841&group_id=5470

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Terry J. Reedy (tjreedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: New class special method lookup change

Initial Comment:

The lookup of special methods invoked implicitly by 
syntax other than explicit instance.attribute changed 
to *not* use __gettattr__ when normal lookup failed.  
This is contrary to docs, which consistently say 
__getattr__ is unchanged.  New special method 
__getattribute__ is also bypassed, contrary to 
implication of doc-essay.

This was reported on c.l.p by Jan Decaluwe using 
2.2.2 on Red Hat Linux.  On Win98, 2.2.1, I get same 
behavior (I added test of __getattribute__):

class Iold:
  def __init__(self,ob):
    self.ob = ob
  def __getattr__(self,name):
    return getattr(self.ob,name)

class Inew(object):
  def __init__(self,ob):
    self.ob = ob
  def __getattr__(self,name):
    return getattr(self.ob,name)

a = Iold(1) #2
b = Inew(1) #2
a.__add__(1) #2
b.__add__(1) #2
a+1 #2
b+1 #error
#TypeError: unsupported operand types for +: 'Inew' 
and 'int'
Inew.__getattribute__ = Inew.__getattr__
b+1 #same error, no loop
#TypeError: unsupported operand types for +: 'Inew' 
and 'int'

b.__add__(1) # WARNING starts 'infinite' loop

def _(self,other): print 'hello'

Inew.__add__ = _
b+1 #prints 'hello', __getattribute__ bypassed.

http://python.org/2.2/descrintro.html says:
"Note that while in general operator overloading 
works just as for classic classes, there are some 
differences. (The biggest one is the lack of support for 
__coerce__; new-style classes should always use the 
new-style numeric API, which passes the other 
operand uncoerced to the __add__ and __radd__ 
methods, etc.) "

Was lookup change meant to be one of differences?

"There's a new way of overriding attribute access. The 
__getattr__ hook, if defined, works the same way as it 
does for classic classes: it is only called if the regular 
way of searching for the attribute doesn't find it."

But it is different.

 "But you can now also override __getattribute__, a 
new operation that is called for all attribute 
references." 

Except for implicit special methods.

I did not classify discrepancy because I don't know 
whether interpreter or docs are off.






----------------------------------------------------------------------

Comment By: Nick Coghlan (ncoghlan)
Date: 2004-09-27 18:54

Message:
Logged In: YES 
user_id=1038590

FWIW, this is still true for Python 2.4 (i.e. the search for
'__add__' by '+' invokes neither __getattr__ nor
__getattribute__).

The 'infinite loop' is courtesy of the call to 'self.ob'
inside __getattr__.

Also of interest - int(b) fails with a TypeError, and str(b)
returns the generic object representation of b, rather than
delegating to the contained object.

So it looks like these methods are not invoked when looking
up any 'magic methods' recognised by the interpreter.
Inspection of PyNumber_Add shows that this is the case - the
objects' method pointers are inspected directly at the C level.

To me, this looks like a documentation bug. Magic methods
are found by checking if the associated slot in the method
structure is not NULL, rather than by actually looking for
the relevant magic 'attribute'.

In order to change the 'implicit' behaviour of the object,
it is necessary to change the contents of the underlying
method slot - a task which is carried out by type whenever
an attribute is set, or when a new instance of type is created.





----------------------------------------------------------------------

Comment By: Terry J. Reedy (tjreedy)
Date: 2002-11-26 14:25

Message:
Logged In: YES 
user_id=593130

Print-style 'debugging' output provided by Bengt Richter in 
a follow-up in the c.l.p. thread "Issue with new-style 
classes and operators" showed that 'a+1' worked because 
of a __getattr__ call getting '__coerce__' (rather 
than '__add__') and that 'b+1' did not trigger such a call.  
So I presume the quoted parenthesized statement about 
__coerce__ and 'new-style numeric API' was meant to 
explain as least this part of the change in behavior.  
However, it is still not clear to me, even after reading the 
development (2.3a) version of the ref manual, why failure 
to find '__add__' in 'the usual places' (to quote RefMan 
3.3.2 __getattr__ entry) does not trigger a call to 
__getattr__, nor why the initial attempt to find it did not 
trigger __getattribute__.  The sentence 'For objects x and y, 
first x.__op__(y) is tried' (3.3.7) implies to me that there is 
an attempt to find __op__ as an attribute of x.  Unless I 
missed something I should have found, a bit more 
clarification might be usefully added.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=643841&group_id=5470