[Python-Dev] Comments: PEP 252, PEP 253 and jython issues

[Guido van Rossum]

> Thanks for the answers, I'm happy the questions were parseable,
> Here are some comments, I think they may be meanigful also for CPython
> 
> > > The question is it possible for CSpecialList for the code that define
> > > for example sq_ass_slice to use sq_slice code in a safe manner?
> > > (AFAIK listobject does something similar) 
> > 
> > I'm not 100% sure I understand this question, but CSpecialList can
> > certainly override sq_ass_slice.  Then, if it wants to call
> > self.__slice__ (in Python terms) it could call
> > obj->ob_type->tp_as_sequence->sq_slice -- this would call C's
> > obj->ob_type->tp_as_sequence->__slice__ if it had one.

> I overlooked this, I think we can implement that also in Jython.
> 
> An important point: should that be the only way to use a Python level
> exposed and overridable method from C code, is that left
> to the programmer ?

I'm not sure I understand this question.  Can you give an example?  I
don't expect that calling into Python from C will be very common.
There are ways to call any Python object from C if you need to.

> The problem is that with the new mro possibly independently C defined 
> things  get interleaved with Python level code, 
> so I imagine that the effect should be a bit disciplined ...

I don't expect this to happen regularly.  if you count on this I think
you're on really thin ice.

> I saw that list_ass_slice calls list_slice directly, is that OK?

It's certainly safe -- any subclass of list will share the same
structure lay-out (this is enforced by the subclassing machinery) so
the list_slice call will find the object properly inititialized.

There are lots of places where the C code doesn't bother to look for
overridden methods -- the PyDict_* API is a common example.  I have no
intention of fixing this.

> > > And how can safely CSpecialList invoke a  "super" behaviour, it is saf
> > > for it to refer to the behaviour offered by list. In principle
> > > given the mro e.g. a __getslice__ redefined in B should shadow
> > > such a behaviour?
> > 
> > I don't know if cooperative super calls are possible at the C level.
> > I certainly didn't define an API for this.  I didn't define a Python
> > level API either, but I know how to o it and it can even be coded in
> > Python.  Hm, I guess that makes it a theoretical possibility at the C
> > level.
> > 
> > I don't expect this to be a common use pattern though (famous last
> > words :-).
> 
> But it is certainely a pattern that a writer of Jython types would expect
> to use. The problem is still that C/Java stuff with the new mro
> get interleaved with Python level behaviour.

If you think this will be a problem in Jython, you may have to provide
a fast efficient Java-level "cooperative super" operation.

> IMHO you would better discipline this and also the above point <wink>.

What do you mean by "discipline"?

> I don't know if there is some GUI toolkit that implement inheritance
> using C and not some language enforcing OO like C++, etc.

Sorry, I don't understand the relevance of this?

> Other OO languages like Smalltalk, Self, or Java allow only to define
> primitives or single methods to add lowel-level functionality,
> they don't try to allow subtyping at that level, the previous
> scenario in Python where you could define just opaque types,
> and in particular your C code never/normally got interleaved with Python
>  behaviour was on that line.

Well, if you like that better, maybe I should remove the instructions
for subclassing at the C level? :-)

My intention is to make the maximum number of different paradigms
usable in Python.  You can do cooperative multiple inheritance, but
only of all your classes are designed with cooperation in mind.  If
some class uses C-level (or Java-level) inheritance and is not written
cooperatively, you have to be a little careful using it as a base
class in a cooperatively written class.  I don't want to enforce
cooperative coding at the Python level either: the
"BaseClass.method(self, args)" approach is still valid, but you have
to beware of the consequences.

> At worst you could say that you allow subtyping but not code sharing,
> but then even a C method for an object should be very careful using
> another overridable method of the very same object.

I definitely have to support code sharing.  I think subclassing at the
C level is useful to create variations of built-in objects with
additional properties, e.g. numbers with additional operations or
formatting parameters, dictionaries with certain restrictions on the
keys, etc.  I don't think such subclasses should be used in complex
multiple inheritance lattices.

> OK you could say: that's C level, everybody should care for herself,
> but with Java we cannot do the same.

Why not?

> > > It would make sense to use Java subclassing to implement 
> > > type subclassing, at least at layout level this does not clash with
> > > the multiple inheritance rule in best_base.
> > 
> > So you would rule out multiple inheritance of types?  Fine with me,
> > but you may regret this a few yearsfrom now (when we all code multiple
> > inheritance all the time :-).

> No it is ruled out at Java level, not Jython level.
> But it is not also ruled out in CPython at C level?

Mostly because I don't provide a way to create multiple base classes.
That would not be hard to add though -- if there's demand.

> > > But than there is the issue of method resolution order: from the
> > > viewpoint of Python code we can implement anything, not that
> > > easy ...
> > > 
> > > But the at the Java level, where we construct types, the
> > > codebase uses the normal single inheritance of java and the
> > > codebase is full of super.foo invocations and of methods that
> > > cross-call each other (potentially overriden) versions, and this
> > > happen also for behaviour that correspond to the slots/ __foo__
> > > special methods of CPython.
> > > 
> > > That's why the two questions are important?
> > 
> > I see.  CPython doesn't have this problem (yet) because it doesn't use
> > inheritance at the C level.

> see above.
> 
> >  I propose that you just try to live with this.

> Related e.g. to Zope porting projects there have been a lot of
> pressure on jython-dev regarding how to code new types and
> metaclasses, A Jython type writer expect to be able to use Java
> naturally (i.e. use inheritance), for that, so the situation isn't
> that easy, is not even easy with the current codebase, because given
> the new mro one can even put some Python behaviour between:
> 
> PyList PythonLevelCode and PyObject :-( (*)

Just sa no. :-)

> >  A safe rule would be to require that all Python classes in a
> > given inheritance graph should derive from the same C/Java class.  I

> That would mean that all C/Java behaviour is always only at the top
> of the mro, right?

If you mean at the end, yes.

> I had the same idea because this would make my life easier.

Sounds good to me.

> A problem: some of the Zope porters reported that there are classes
> that inherits from more than a single ExtensionClass in Zope :-(
> don't know if that is true. (**)

Me neither.  But the problem is probably shallow -- Zope mostly uses
mixins of various sorts, so these classes probably don't override
stuff -- they just define new methods.  (I haven't looked at what it
would take to replace ExtensionClass with the new metaclasses, but
that's definitely on the agenda.)

> > don't know if we should enforce this or just warn about it in the
> > documentation.

> It would make our (jython-dev) life easier and avoid some impredictable
>  problem even with CPython and code sharing,
>   but see previous note (**)

So enforce it and see how it turns out in practice.

> >  I don't want to force you to call a cooperative super
> > method at the Java level -- it would be very slow, I suspect...

> Yes ... very slow and we would have to use that also at the very core
> of the hierarchy, see (*)
> but the problem is more complicated, in Java given three classes
> 
> C extends B extends A
> and a method m
> C.m overrides B.m overrides A.m
> 
> there is not direct way (even using reflection) 
> to apply B.m behaviour to a C object, unless
> the programmer has left some hook in C using super
> or in B ( a method B_m e.g.).

I guess C.m has to call super.m to invoke B.m, right?

Does this mean that the following can't be made to work?

class C(list):
    def add_spam(self):
        return list.append(self, "spam")

It's legal Python (although in general questionable style of course).

> ... complicated :(

:-(

--Guido van Rossum (home page: http://www.python.org/~guido/)