[Python-Dev] instancemethod_getattro seems to be partially wrong

[Guido van Rossum]

> Guido van Rossum wrote:
> > Summary: Chistian is right after all.  instancemethod_getattro should
> > always prefer bound method attributes over function attributes.

[Christian]
> Guido, I'm very happy with your decision, which is most
> probably a wise decision (without any relation to me).
> 
> The point is, that I didn't know what's right or wrong,
> so basically I was asking for advice on a thing I felt
> unhappy with. So I asked you to re-think if the behavior
> is really what you indented, or if you just stopped, early.
> 
> Thanks a lot!
> 
> That's the summary and all about it, you can skip the rest if you like.

Note to python-dev folks: I will make the change in 2.4.  I won't
backport to 2.3 unless someone can really make a case for it; it
*does* change behavior.

[...]
> > The *intention* was for the 2.2 version to have the same behavior:
> > only im_func, im_self and im_class would be handled by the bound
> > method, other attributes would be handled by the function object.
> 
> Ooh, I begin to understand!
> 
> > This is what the IsData test is attempting to do -- the im_*
> > attributes are represented by data descriptors now.  The __class__
> > attribute is also a data descriptor, so that C().x.__class__ gives us
> > <type 'instancemethod'> rather than <type 'function'>.
> 
> IsData is a test for having a write method, too, so we have
> the side effect here that im_* works like I expect, since
> they happen to be writable?
> Well, I didn't look into 2.3 for this, but in 2.2 I get
> 
>  >>> a().x.__class__=42
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> TypeError: __class__ must be set to new-style class, not 'int' object
> [9511 refs]
>  >>>
> 
> which says for sure that this is a writable property, while
> 
>  >>> a().x.im_class=42
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> TypeError: readonly attribute
> [9511 refs]
>  >>>
> 
> seems to be handled differently.
> 
> I only thought of IsData in terms of accessing the
> getter/setter wrappers.

It's all rather complicated.  IsData only checks for the presence of a
tp_descr_set method in the type struct.  im_* happen to be implemented
by a generic approach for defining data attributes which uses a
descriptor type that has a tp_descr_set method, but its implementation
looks for a "READONLY" flag.  This is intentional -- in fact, having a
tp_descr_set (or __set__) method that raises an error is the right way
to create a read-only data attribute (at least for classes whose
instances have a __dict__).

[...]
> I don't need to pickle classes, this works fine in most cases,
> and behavior can be modified by users.

Right.  When you are pickling classes, you're really pickling code,
not data, and that's usually not what pickling is used for.  (Except
in Zope 3, which can store code in the database and hence must pickle
classes.  But it's a lot of work, as Jeremy can testify. :-)

> > (I wonder if the pickling code shouldn't try to call
> > x.__class__.__reduce__(x) rather than x.__reduce__() -- then none of
> > these problems would have occurred... :-)
> 
> That sounds reasonable. Explicit would have been better than
> implicit (by hoping for the expected bound chain).

Especially since *internally* most new-style classes do this for all
of the built-in operations (operations for which there is a function
pointer slot in the type struct or one of its extensions).  This is
different from old-style classes: a classic *instance* can overload
(nearly) any special method by having an instance attribute,
e.g. __add__; but this is not supported for new-style instances.

> __reduce__ as a class method would allow to explicitly spell
> that I want to reduce the instance x of class C.
> 
> x.__class__.__reduce__(x)
> 
> While, in contrast
> 
> x.__class__.__reduce__(x.thing)
> 
> would spell that I want to reduce the "thing" property of the
> x instance of C.
> 
> While
> 
> x.__class__.__reduce__(C.thing)  # would be the same as
> C.__reduce__(C.thing)
> 
> which would reduce the class method "thing" of C, or the class
> property of C, or whatsoever of class C.

You've lost me here.  How does x.__class__.__reduce__ (i.e.,
C.__reduce__) tell the difference between x and x.thing and C.thing???

> I could envision a small extension to the __reduce__ protocol,
> by providing an optional parameter, which would open these
> new ways, and all pickling questions could be solved, probably.
> This is so, since we can find out whether __reduce__ is a class
> method or not.
> If it is just an instance method (implictly bound), it behaves as
> today.
> If it is a class method, is takes a parameter, and then it can find
> out whether to pickle a class, instance, class property or an instance
> property.
> 
> Well, I hope. The above was said while being in bed with 39° Celsius,
> so don't put my words on the assay-balance.

I sure don't understand it.  If you really want this, please sit down
without a fever and explain it with more examples and a clarification
of what you want to change, and how.

[...]
> Until now, I only had to change traceback.c and iterator.c, since
> these don't export enough of their structures to patch things
> from outside. If at some point somebody might decide that some of
> this support code makes sense for the main distribution, things
> should of couzrse move to where they belong.

Do you realize that (in C code) you can always get at a type object if
you can create an instance of it, and then you can patch the type
object just fine?

[...]
> What I want to do at some time is to change cPickle to use
> a non-recursive implementation. (Ironically, the Python pickle
> engine *is* non-recursive, if it is run under Stackless).
> So, if I would hack at cPickle at all, I would probably do the
> big big change, and that would be too much to get done in
> reasonable time. That's why I decided to stay small and just
> chime a few __reduce__ thingies in, for the time being.
> Maybe this was not the best way, I don't know.

What's the reason for wanting to make cPickle non-recursive?

[...]
> Right. probably, I will get into trouble with pickling
> unbound class methods.
> Maybe I would just ignore this. Bound class methods do
> appear in my Tasklet system and need to get pickled.
> Unbound methods are much easier to avoid and probably
> not worth the effort. (Yes, tomorrow I will be told
> that it *is* :-)

Unbound methods have the same implementation as bound methods -- they
have the same type, but im_self is None (NULL at the C level).  So you
should be able to handle this easily.  (Unbound methods are not quite
the same as bare functions; the latter of course are pickled by
reference, like classes.)

[...]
> That means, for Py 2.2 and 2.3, my current special case for
> __reduce__ is exactly the way to go, since it doesn't change any
> semantics but for __reduce__, and in 2.4 I just drop these
> three lines? Perfect!

Right.  (I'm not quite ready for the 2.4 checkin, watch the checkins
list though.)

--Guido van Rossum (home page: http://www.python.org/~guido/)