[Python-Dev] instancemethod_getattro seems to be partially wrong

Guido van Rossum guido at python.org
Thu Nov 20 01:18:45 EST 2003

Summary: Chistian is right after all.  instancemethod_getattro should
always prefer bound method attributes over function attributes.

> >>No no no, I'm not fiddling around with any internals, here.
> >>I just want to use the machinary as it is, and to be able to
> >>pickle almost everything.
> Sorry, this was a lie.


OK, you're forgiven.

> Sure I'm fiddling internaly, but simply by
> installing some __reduce__ methids, hoping that
> they work.

OK, so you *could* just make the change you want, but you are asking
why it isn't like that in the first place.  Good idea...

> This worked most of the time, but I'm having problems
> with bound methods.

We've established that without a doubt, yes. :-)

> > But how, I wonder, are you providing it?  You can't subclass
> > instancemethod -- how do you manage to add a __reduce__ method to it
> > without fiddling with any internals?
> I added __reduce__ to the PyMethod type and tried to figure out
> why it didn't take it.

OK.  Stating that upfront would have helped...

> >>My expectation is that C().x.__reduce__ gives me the bound
> >>__reduce__ method of the bound x method of a C instance.
> > 
> > 
> > Yes, unfortunately you get the __reduce__ method of the unbound
> > function instead.
> > 
> > I think Martin is right: copy_reg may be your last hope.  (Or
> > subclassing pickle to special-case instancemethod.)
> Well, I see your point, but please let me explain mine, again:
> If there is a class C which has a method x, then C().x is
> a perfectly fine expression, yielding a bound method.

Of course.

> If I now like to pickle this expression, I would use the
> __reduce__ protocol and ask C().x for its __reduce__ property.

Which unfortunately gets the __reduce__ property of the underlying
*function* object (also named x) used to implement the method.  This
function can be accessed as C.__dict__['x'].  (Not as C.x, that
returns an unbound method object, which is the same kind of object as
a bound method object but without an instance. :-)

> Now, please see that __reduce__ has no parameters, i.e. it has
> no other chance to do the right thing(TM) but by relying
> on to be bound to the right thing.
> So, doesn't it make sense to have __reduce__ to be always returned
> as a method of some bound anything?
> In other words, shouldn't things that are only useful as bound
> things, always be bound?

This question doesn't address the real issue, which is the attribute
delegation to the underlying function object.

What *should* happen when the same attribute name exists on the
function and on the bound method?  In 2.1, when function attributes
were first introduced, this was easy: a few attributes were special
for the bound method (im_func, im_self, im_class) and for these the
bound method attribute wins (if you set an attribute with one of those
names on the function, you can't access it through the bound method).

The *intention* was for the 2.2 version to have the same behavior:
only im_func, im_self and im_class would be handled by the bound
method, other attributes would be handled by the function object.
This is what the IsData test is attempting to do -- the im_*
attributes are represented by data descriptors now.  The __class__
attribute is also a data descriptor, so that C().x.__class__ gives us
<type 'instancemethod'> rather than <type 'function'>.

But for anything else, including the various methods that all objects
inherit from 'object' unless they override them, the choice was made
to let the function attribute win.

But when we look at the attributes where both function and bound
method provide a value, it seems that the bound method's offering is
always more useful!  You've already established this for __reduce__;
the same is true for __call__ and __str__, and there I stopped.
(Actually, I also looked at __setattr__, where delegation to the
function also seems a mistake: C().x.foo = 42 is refused, but
C().x.__setattr__('foo', 42) sets the attribute on the function,
because this returns the (bound) method __setattr__ on functions.)

> > The pickling machinery wasn't intended to pickle bound methods or
> > functions etc., and doesn't particularly go out of its way to allow
> > you to add that functionality.
> The pickling machinery gives me an __reduce__ interface, and I'm
> expecting that this is able to pickle everything.

I don't think you'd have a chance of pickle classes if you only relied
on __reduce__.  Fortunately there are other mechanisms. :-)

(I wonder if the pickling code shouldn't try to call
x.__class__.__reduce__(x) rather than x.__reduce__() -- then none of
these problems would have occurred... :-)

> ...
> > And it would be except for the delegation of method attributes to
> > function attributes.  It is a similar aliasing problem as you see when
> > you try to access the __getattr__ implementation for classes as
> > C.__getattr__ -- you get the __getattr__ for C instances instead.  So
> > you have to use type(C).__getattr__ instead.  That would work for
> > __reduce__ too I think: new.instancemethod.__reduce__(C().f).
> I agree!
> But I can't do this in this context, using __reduce__ only.
> In other words, I'd have to add stuff to copyreg.py, which
> I tried to circumvent.

Or you could change the pickling system.  Your choice of what to
change and what not to change seems a bit arbitrary. :-)

> ...
> > OK, so you *are* messing with internals after all (== changing C
> > code), right?  Or else how do you accomplish this?
> Yessir, I'm augmenting all things-to-be-pickled with __reduce__
> methods. And this time is the first time that it doesn't work.

But not necessarily the last time. :-)

> ...
> > I have a real job too, that's why I have little time to help you. :-(
> I agree (and I didn't ask *you* in the first place), but still
> I'd like to ask the general question:
> Is this really the right way to handle bound objects?
> Is the is_data criterion correct?
> If I am asking for an attribute that makes *only* sense if it is
> bound, like in the parameter-less __reduce__ case, wouldn't
> it be the correct behavior to give me that bound object?
> I have the strong impression that there is some difference
> in methods which isn't dealt with, correctly, at the moment.
> If a method wants to be bound to something, it should be
> get bound to something.
> Especially, if this method is useless without being bound.

It's not that it isn't being bound.  It's that the *wrong* attribute
is being bound (the function's __reduce__ method, bound to the
function object, is returned!).

> Please, swallow this idea a little bit, before rejecting
> it. I think that "is_data" is too rough and doesn't fit
> the requirements, all the time.

I agree.  The bound method's attributes should always win, since bound
methods only have a small, fixed number of attributes, and they are
all special for bound methods.

This *is* a change in functionality, even though there appear to be no
unit tests for it, so I'm reluctant to fix it in 2.3.  But I think in
2.4 it should definitely change.

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list