[ python-Bugs-729913 ] metaclasses, __getattr__, and special methods

[SourceForge.net]

Bugs item #729913, was opened at 2003-04-29 23:57
Message generated for change (Comment added) made by arigo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=729913&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Type/class unification
Group: None
>Status: Closed
>Resolution: Invalid
Priority: 5
Submitted By: Bjorn Pettersen (bpettersen)
Assigned to: Nobody/Anonymous (nobody)
Summary: metaclasses, __getattr__, and special methods

Initial Comment:
__getattr__ on metaclasses aren't called when it would 
seem "logical" <wink> for it to be. E.g.:

>>> class meta(type):
...   def __getattr__(cls, name):
...     if name == '__len__':
...        print "meta.__getattr__('__len__')"
...        return lambda: 42
...     else:
...        print 'meta.__getattr__', name
...        return name
...
>>> class S(object):
...   __metaclass__ = meta
...
>>> S.__len__()
meta.__getattr__('__len__')
42
>>> len(S)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: len() of unsized object
>>>

I was told that special method "foo(x, arg)" was 
implemented as
"type(x).__foo__(x, arg)", which doesn't seem to be the 
case always...
Compare:

>>> class meta(type):
...   def __len__(cls):
...     return 42
...
>>> class S(object):
...   __metaclass__ = meta
...
>>> S.__len__()
42
>>> len(S)
42
>>>

So, it looks like it's looking up __len__ in the 
metaclass, but not
falling back on __getattr__ when it isn't there? I've 
looked at the C
code and it seems like special methods each have their 
own way of
finding the function they're needing.

>From Alex Martelli:
Ah yes, I see now!  Yes, functions such as len() rely on 
slots in
the type object, e.g. as you've noticed:

> finding the function they're needing, e.g. for len, it 
looks like it
> uses:
> 
>   m = o->ob_type->tp_as_sequence;
>   if (m && m->sq_length)
>     return m->sq_length(o);
> 
>   return PyMapping_Size(o);

and the "incredibly complex thinking" (quoting from 
typeobject.c)
in update_one_slot doesn't seem to work except for 
operations the
which "the class overrides in its dict" (again from a 
comment in
typeobject.c, this one for fixup_slot_dispatchers).

The issue may be with _PyType_Lookup (again in the 
same ,c file),
which just gives up if it can't find a name somewhere 
along the mro
(it doesn't "look upwards" to the metaclass) while 
type_getattro
DOES work upwards on the metaclass too.  Hmmmm.  
I'm not sure I
really understand all that's going on here - it IS a rather 
hairy
zone of the code.  Maybe you can post this as a bug in 
2.3 beta 1
on sourgeforge (ideally showing where in the docs it 
defines the
semantics that it then doesn't respect) so we can get 
this looked
at by the few people who really DO grasp these parts...;-
).  There
is probably some sound implementation reason for the 
current
behavior, but if so it should be better documented, I 
think.

Back to me:
The point being that I haven't found any place in the 
documentation that defines what the attribute lookup is 
on new-style classes (and the C code is too hairy for 
me to understand :-)

As a special case of this problem, super() can't create 
an object which intercepts the special methods like it 
does for other methods, e.g.:

  super(MyCls, self).__getitem__(5)

works, but not

  super(MyCls, self)[5]

I don't know if that is intended or not, but it's not 
documented, though neither is exactly _what_ super is? 
(i.e. it looks like it's an object, that when you call a 
method, 'm', on it, uses the superclass method 'm', but 
the subclass versions of all other methods, although as 
above, not in all contexts, and I'm not sure whether 
you're supposed to be able to treat it as a first class 
object [pass as arg, return, etc])....

-- bjorn



----------------------------------------------------------------------

>Comment By: Armin Rigo (arigo)
Date: 2005-12-26 17:13

Message:
Logged In: YES 
user_id=4771

This is a known documentation bug: all this is expected, but
under-documented.  Indeed, len(x) calls the special method
__len__ of 'x', but what is not specified is the real
definition of "calling a special method" on an object 'x':
it is to look up the name "__len__" in the dict of type(x),
then in the dict of the parent types in MRO order.  It's
really not the same thing as an attribute lookup.

----------------------------------------------------------------------

Comment By: Reinhold Birkenfeld (birkenfeld)
Date: 2005-10-01 13:43

Message:
Logged In: YES 
user_id=1188172

I closed #789262 as a duplicate of this one. More info may
be there.

----------------------------------------------------------------------

Comment By: Michael Hudson (mwh)
Date: 2004-11-15 07:20

Message:
Logged In: YES 
user_id=6656

You could try 

http://starship.python.net/crew/mwh/hacks/oop-after-python22.txt

(or attach pdf to the end instead...)

You say:

> The point being that I haven't found any place in the 
> documentation that defines what the attribute lookup is 
> on new-style classes

That's not the problem -- attribute lookup is fairly easy.  What 
you're missing is that attribute lookup != special method lookup.

This probably should be in the core documentation, yes.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=729913&group_id=5470