Re: [Python-Dev] Comments: PEP 252, PEP 253 and jython issues
Thanks for the answers, I'm happy the questions were parseable, Here are some comments, I think they may be meanigful also for CPython
The question is it possible for CSpecialList for the code that define for example sq_ass_slice to use sq_slice code in a safe manner? (AFAIK listobject does something similar)
I'm not 100% sure I understand this question, but CSpecialList can certainly override sq_ass_slice. Then, if it wants to call self.__slice__ (in Python terms) it could call obj->ob_type->tp_as_sequence->sq_slice -- this would call C's obj->ob_type->tp_as_sequence->__slice__ if it had one. I overlooked this, I think we can implement that also in Jython.
An important point: should that be the only way to use a Python level exposed and overridable method from C code, is that left to the programmer ? The problem is that with the new mro possibly independently C defined things get interleaved with Python level code, so I imagine that the effect should be a bit disciplined ... I saw that list_ass_slice calls list_slice directly, is that OK?
And how can safely CSpecialList invoke a "super" behaviour, it is saf for it to refer to the behaviour offered by list. In principle given the mro e.g. a __getslice__ redefined in B should shadow such a behaviour?
I don't know if cooperative super calls are possible at the C level. I certainly didn't define an API for this. I didn't define a Python level API either, but I know how to o it and it can even be coded in Python. Hm, I guess that makes it a theoretical possibility at the C level.
I don't expect this to be a common use pattern though (famous last words :-).
But it is certainely a pattern that a writer of Jython types would expect to use. The problem is still that C/Java stuff with the new mro get interleaved with Python level behaviour. IMHO you would better discipline this and also the above point <wink>. I don't know if there is some GUI toolkit that implement inheritance using C and not some language enforcing OO like C++, etc. Other OO languages like Smalltalk, Self, or Java allow only to define primitives or single methods to add lowel-level functionality, they don't try to allow subtyping at that level, the previous scenario in Python where you could define just opaque types, and in particular your C code never/normally got interleaved with Python behaviour was on that line. At worst you could say that you allow subtyping but not code sharing, but then even a C method for an object should be very careful using another overridable method of the very same object. OK you could say: that's C level, everybody should care for herself, but with Java we cannot do the same.
It would make sense to use Java subclassing to implement type subclassing, at least at layout level this does not clash with the multiple inheritance rule in best_base.
So you would rule out multiple inheritance of types? Fine with me, but you may regret this a few yearsfrom now (when we all code multiple inheritance all the time :-). No it is ruled out at Java level, not Jython level. But it is not also ruled out in CPython at C level?
But than there is the issue of method resolution order: from the viewpoint of Python code we can implement anything, not that easy ...
But the at the Java level, where we construct types, the codebase uses the normal single inheritance of java and the codebase is full of super.foo invocations and of methods that cross-call each other (potentially overriden) versions, and this happen also for behaviour that correspond to the slots/ __foo__ special methods of CPython.
That's why the two questions are important?
I see. CPython doesn't have this problem (yet) because it doesn't use inheritance at the C level. see above.
I propose that you just try to live with this. Related e.g. to Zope porting projects there have been a lot of pressure on jython-dev regarding how to code new types and metaclasses, A Jython type writer expect to be able to use Java naturally (i.e. use inheritance), for that, so the situation isn't that easy, is not even easy with the current codebase, because given the new mro one can even put some Python behaviour between:
PyList PythonLevelCode and PyObject :-( (*)
A safe rule would be to require that all Python classes in a given inheritance graph should derive from the same C/Java class. I That would mean that all C/Java behaviour is always only at the top of the mro, right?
don't know if we should enforce this or just warn about it in the documentation. It would make our (jython-dev) life easier and avoid some impredictable
I had the same idea because this would make my life easier. A problem: some of the Zope porters reported that there are classes that inherits from more than a single ExtensionClass in Zope :-( don't know if that is true. (**) problem even with CPython and code sharing, but see previous note (**)
I don't want to force you to call a cooperative super method at the Java level -- it would be very slow, I suspect... Yes ... very slow and we would have to use that also at the very core of the hierarchy, see (*) but the problem is more complicated, in Java given three classes
C extends B extends A and a method m C.m overrides B.m overrides A.m there is not direct way (even using reflection) to apply B.m behaviour to a C object, unless the programmer has left some hook in C using super or in B ( a method B_m e.g.). ... complicated :( regards.
Thanks for the answers, I'm happy the questions were parseable, Here are some comments, I think they may be meanigful also for CPython
The question is it possible for CSpecialList for the code that define for example sq_ass_slice to use sq_slice code in a safe manner? (AFAIK listobject does something similar)
I'm not 100% sure I understand this question, but CSpecialList can certainly override sq_ass_slice. Then, if it wants to call self.__slice__ (in Python terms) it could call obj->ob_type->tp_as_sequence->sq_slice -- this would call C's obj->ob_type->tp_as_sequence->__slice__ if it had one.
I overlooked this, I think we can implement that also in Jython.
An important point: should that be the only way to use a Python level exposed and overridable method from C code, is that left to the programmer ?
I'm not sure I understand this question. Can you give an example? I don't expect that calling into Python from C will be very common. There are ways to call any Python object from C if you need to.
The problem is that with the new mro possibly independently C defined things get interleaved with Python level code, so I imagine that the effect should be a bit disciplined ...
I don't expect this to happen regularly. if you count on this I think you're on really thin ice.
I saw that list_ass_slice calls list_slice directly, is that OK?
It's certainly safe -- any subclass of list will share the same structure lay-out (this is enforced by the subclassing machinery) so the list_slice call will find the object properly inititialized. There are lots of places where the C code doesn't bother to look for overridden methods -- the PyDict_* API is a common example. I have no intention of fixing this.
And how can safely CSpecialList invoke a "super" behaviour, it is saf for it to refer to the behaviour offered by list. In principle given the mro e.g. a __getslice__ redefined in B should shadow such a behaviour?
I don't know if cooperative super calls are possible at the C level. I certainly didn't define an API for this. I didn't define a Python level API either, but I know how to o it and it can even be coded in Python. Hm, I guess that makes it a theoretical possibility at the C level.
I don't expect this to be a common use pattern though (famous last words :-).
But it is certainely a pattern that a writer of Jython types would expect to use. The problem is still that C/Java stuff with the new mro get interleaved with Python level behaviour.
If you think this will be a problem in Jython, you may have to provide a fast efficient Java-level "cooperative super" operation.
IMHO you would better discipline this and also the above point <wink>.
What do you mean by "discipline"?
I don't know if there is some GUI toolkit that implement inheritance using C and not some language enforcing OO like C++, etc.
Sorry, I don't understand the relevance of this?
Other OO languages like Smalltalk, Self, or Java allow only to define primitives or single methods to add lowel-level functionality, they don't try to allow subtyping at that level, the previous scenario in Python where you could define just opaque types, and in particular your C code never/normally got interleaved with Python behaviour was on that line.
Well, if you like that better, maybe I should remove the instructions for subclassing at the C level? :-) My intention is to make the maximum number of different paradigms usable in Python. You can do cooperative multiple inheritance, but only of all your classes are designed with cooperation in mind. If some class uses C-level (or Java-level) inheritance and is not written cooperatively, you have to be a little careful using it as a base class in a cooperatively written class. I don't want to enforce cooperative coding at the Python level either: the "BaseClass.method(self, args)" approach is still valid, but you have to beware of the consequences.
At worst you could say that you allow subtyping but not code sharing, but then even a C method for an object should be very careful using another overridable method of the very same object.
I definitely have to support code sharing. I think subclassing at the C level is useful to create variations of built-in objects with additional properties, e.g. numbers with additional operations or formatting parameters, dictionaries with certain restrictions on the keys, etc. I don't think such subclasses should be used in complex multiple inheritance lattices.
OK you could say: that's C level, everybody should care for herself, but with Java we cannot do the same.
Why not?
It would make sense to use Java subclassing to implement type subclassing, at least at layout level this does not clash with the multiple inheritance rule in best_base.
So you would rule out multiple inheritance of types? Fine with me, but you may regret this a few yearsfrom now (when we all code multiple inheritance all the time :-).
No it is ruled out at Java level, not Jython level. But it is not also ruled out in CPython at C level?
Mostly because I don't provide a way to create multiple base classes. That would not be hard to add though -- if there's demand.
But than there is the issue of method resolution order: from the viewpoint of Python code we can implement anything, not that easy ...
But the at the Java level, where we construct types, the codebase uses the normal single inheritance of java and the codebase is full of super.foo invocations and of methods that cross-call each other (potentially overriden) versions, and this happen also for behaviour that correspond to the slots/ __foo__ special methods of CPython.
That's why the two questions are important?
I see. CPython doesn't have this problem (yet) because it doesn't use inheritance at the C level.
see above.
I propose that you just try to live with this.
Related e.g. to Zope porting projects there have been a lot of pressure on jython-dev regarding how to code new types and metaclasses, A Jython type writer expect to be able to use Java naturally (i.e. use inheritance), for that, so the situation isn't that easy, is not even easy with the current codebase, because given the new mro one can even put some Python behaviour between:
PyList PythonLevelCode and PyObject :-( (*)
Just sa no. :-)
A safe rule would be to require that all Python classes in a given inheritance graph should derive from the same C/Java class. I
That would mean that all C/Java behaviour is always only at the top of the mro, right?
If you mean at the end, yes.
I had the same idea because this would make my life easier.
Sounds good to me.
A problem: some of the Zope porters reported that there are classes that inherits from more than a single ExtensionClass in Zope :-( don't know if that is true. (**)
Me neither. But the problem is probably shallow -- Zope mostly uses mixins of various sorts, so these classes probably don't override stuff -- they just define new methods. (I haven't looked at what it would take to replace ExtensionClass with the new metaclasses, but that's definitely on the agenda.)
don't know if we should enforce this or just warn about it in the documentation.
It would make our (jython-dev) life easier and avoid some impredictable problem even with CPython and code sharing, but see previous note (**)
So enforce it and see how it turns out in practice.
I don't want to force you to call a cooperative super method at the Java level -- it would be very slow, I suspect...
Yes ... very slow and we would have to use that also at the very core of the hierarchy, see (*) but the problem is more complicated, in Java given three classes
C extends B extends A and a method m C.m overrides B.m overrides A.m
there is not direct way (even using reflection) to apply B.m behaviour to a C object, unless the programmer has left some hook in C using super or in B ( a method B_m e.g.).
I guess C.m has to call super.m to invoke B.m, right? Does this mean that the following can't be made to work? class C(list): def add_spam(self): return list.append(self, "spam") It's legal Python (although in general questionable style of course).
... complicated :(
:-( --Guido van Rossum (home page: http://www.python.org/~guido/)
participants (2)
-
Guido van Rossum -
Samuele Pedroni