getting special from type, not instance (was Re: [Python-Dev] copy confusion)

[Alex Martelli]

Since this bug isn't the cause of Fredrik's problem I'm changing the 
subject (and keep discussing the specific problem that Fredrik 
uncovered under the original subject).

On 2005 Jan 12, at 05:11, Guido van Rossum wrote:
    ...
>> I had exactly the same metabug in the pep 246 reference 
>> implementation,
>> Armin Rigo showed how to fix it in his only recent post.
>
> Don't recall seeing that, but if you or he can fix this without
> breaking other stuff, it's clear you should go ahead. (This worked in
> 2.2, FWIW; it broke in 2.3.)

Armin's fix was to change:

    conform = getattr(type(obj), '__conform__', None)

into:

    for basecls in type(obj).__mro__:
        if '__conform__' in basecls.__dict__:
            conform = basecls.__dict__['__conform__']
            break
    else:
        # not found

I have only cursorily examined the rest of the standard library, but it 
seems to me there may be a few other places where getattr is being used 
on a type for this purpose, such as pprint.py which has a couple of 
occurrences of
     r = getattr(typ, "__repr__", None)

Since this very same replacement is needed in more than one place for 
"get the following special attribute from the type of the object', it 
seems that a function to do it should be introduced in one place and 
used from where it's needed:

def get_from_first_dict(dicts, name, default=None):
    for adict in dicts:
        try:
            return adict[name]
        except KeyError:
            pass
    return default

to be called, e.g. in the above example with '__conform__', as:

conform = get_from_first_dict(
               (basecls.__dict__ for basecls in type(obj).__mro__),
               '__conform__'
           )

The needed function could of course be made less general, by giving 
more work to the function and less work to the caller, all the way down 
to:

def getspecial(obj, name, default=None):
    for basecls in type(obj).__mro__:
        try:
            return basecls.__dict__[name]
        except KeyError:
            pass
    return default

to be called, e.g. in the above example with '__conform__', as:

conform = getspecial(obj, '__conform__')

This has the advantage of not needing the genexp, so it's usable to 
implement the fix in 2.3.5 as well as in 2.4.1.  Moreover, it can 
specialcase old-style class instances to provide the backwards 
compatible behavior, if desired -- that doesn't matter (but doesn't 
hurt) to fix the bug in copy.py, because in that case old-style 
instances have been already specialcases previously, and might help to 
avoid breaking anything in other similar bugfixes, so that's what I 
would suggest:

def getspecial(obj, name, default=None):
    if isinstance(obj, types.InstanceType):
        return getattr(obj, name, default)
    for basecls in type(obj).__mro__:
        try:
            return basecls.__dict__[name]
        except KeyError:
            pass
    return default

The tradeoff between using type(obj) and obj.__class__ isn't 
crystal-clear to me, but since the latter might apparently be faked by 
some proxy to survive isinstance calls type(obj) appears to me to be 
right.


Where in the standard library to place this function is not clear to me 
either.  Since it's going into bugfix-only releases, I assume it 
shouldn't be "published".  Maybe having it as copy._getspecial (i.e. 
with a private name) is best, as long as it's OK to introduce some 
coupling by having (e.g.) pprint call copy._getspecial too.

Performance might be a problem, but the few bugfix locations where a 
getattr would be replaced by this getspecial don't seem to be hotspots, 
so maybe we don't need to worry about it for 2.3 and 2.4 (it might be 
nice to have this functionality "published" in 2.5, of course, and then 
it should probably be made fast).

Feedback welcome -- the actual patch will doubtless be tiny, but it 
would be nice to have it right the first time (as it needs to go into 
both the 2.3 and 2.4 bugfix branches and the 2.5 head).


Alex