[Python-Dev] super_getattro() Behaviour

[Phil Thompson]

> "Phil Thompson" <phil at riverbankcomputing.co.uk> writes:
>
>> In PyQt, wrapped types implement lazy access to the type dictionary
>> through tp_getattro. If the normal attribute lookup fails, then private
>> tables are searched and the attribute (if found) is created on the fly
>> and
>> returned. It is also put into the type dictionary so that it is found
>> next
>> time through the normal lookup. This is done to speed up the import of,
>> and the memory consumed by, the qt module which contains thousands of
>> class methods.
>>
>> This all works fine - except when super is used.
>>
>> The implementation of super_getattro() doesn't use the normal attribute
>> lookup (ie. doesn't go via tp_getattro). Instead it walks the MRO
>> hierarchy itself and searches instance dictionaries explicitly. This
>> means
>> that attributes that have not yet been referenced (ie. not yet been
>> cached
>> in the type dictionary) will not be found.
>>
>> Questions...
>>
>> 1. What is the reason why it doesn't go via tp_getattro?
>
> Because it wouldn't work if it did?  I'm not sure what you're
> suggesting here.

I'm asking for an explanation for the current implementation. Why wouldn't
it work if it got the attribute via tp_getattro?

>> 2. A possible workaround is to subvert the ma_lookup function of the
>> type
>> dictionary after creating the type to do something similar to what my
>> tp_getattro function is doing.
>
> Eek!

Agreed.

>> Are there any inherent problems with that?
>
> Well, I think the layout of dictionaries is fiercely private.  IIRC,
> the only reason it's in a public header is to allow some optimzations
> in ceval.c (though this isn't at all obvious from the headers, so
> maybe I'm mistaken).

Yes, having looked in more detail at the dict implementation I really
don't want to go there.

>> 3. Why, when creating a new type and eventually calling type_new() is a
>> copy of the dictionary passed in made?
>
> I think this is to prevent changes to tp_dict behind the type's back.
> It's important to keep the dict and the slots in sync.
>
>> Why not take a reference to it?  This would allow a dict sub-class
>> to be used as the type dictionary. I could then implement a
>> lazy-dict sub-class with the behaviour I need.
>
> Well, not really, because super_getattro uses PyDict_GetItem, which
> doesn't respect subclasses...

I suppose I was hoping for more C++ like behaviour.

>> 4. Am I missing a more correct/obvious technique? (There is no need to
>> support classic classes.)
>
> Hum, I can't think of one, I'm afraid.
>
> There has been some vague talk of having a tp_lookup slot in
> typeobjects, so
>
> PyDict_GetItem(t->tp_dict, x);
>
> would become
>
> t->tp_lookup(x);
>
> (well, ish, it might make more sense to only do that if the dict
> lookup fails).

That would be perfect. I can't Google any reference to a discussion - can
you point me at something?

> For now, not being lazy seems your only option :-/ (it's what PyObjC
> does).

Not practical I'm afraid. I think I can only document that super doesn't
work in this context.

Thanks,
Phil