Re: [Python-Dev] [Fwd: Re: PEP 384: Defining a Stable ABI]
My perspective is as follows: 1) If PEP-384 had always been in place, my life would now be a lot easier. 2) Since it hasn't always been in place, its introduction won't help me in the short term: there are an awful lot of extension modules that use excluded functions (for example, all(?) PyCxx modules use PyCode_New and PyFrame_New to get nicer tracebacks), and I'll still have to handle all these cases until everyone is up-to-date with whatever version of Python this gets accepted into. 3) Regardless, this PEP makes me very happy, because I can now look forward to the glorious day when all extension modules are 384-compatible (and even *some* modules becoming compatible will make me pretty happy). However, I'm not sure exactly how we can get there from here; I suspect that certain features of certain extensions already depend critically upon implementation details which will become hidden. The most extreme illustrative example I know is from NumPy (in scalarmathmodule.c), and looks like this: PyInt_Type.tp_as_number = PyLongArrType_Type.tp_as_number; PyInt_Type.tp_compare = PyLongArrType_Type.tp_compare; PyInt_Type.tp_richcompare = PyLongArrType_Type.tp_richcompare; ...and I fear that many many similar (if perhaps less frightening) dependencies exist elsewhere. Regardless, in answer to the two specific questions you ask: a) We don't really have that option. However, I would have a much higher degree of confidence in running PEP-384-compatible modules under Ironclad than I do with current modules, simply because I would no longer need to worry about (say) edge cases in which extension writers suddenly try to directly access op->ob_type->tp_as_number->nb_power. b) I can't think of any more useful restrictions. The PEP would solve my biggest current worry, which is that my current implementation allows managed/unmanaged lists to fall out of sync in certain circumstances (but if every list mutation happened via an API call, it wouldn't be an issue). Best Regards William Michael Foord wrote:
The questions from Martin v. Lowis are in the email below.
The PEP under discussion is:
http://www.python.org/dev/peps/pep-0384/
I can proxy any replies you want to send, or you can join Python-dev.
All the best,
Michael
-------- Original Message -------- Subject: Re: [Python-Dev] PEP 384: Defining a Stable ABI Date: Mon, 18 May 2009 08:00:57 +0200 From: "Martin v. Löwis"
To: Michael Foord CC: Dino Viehland , Python-Dev , Unladen Swallow , Python List References: <4A107988.3020202@v.loewis.de> <4A108ABF.9060909@v.loewis.de> <1A472770E042064698CB5ADC83A12ACD016E82D6@TK5EX14MBXC116.redmond.corp.microsoft.com> <4A1097F5.2050009@v.loewis.de> <4A10A368.1060406@voidspace.org.uk> It also might make it easier for alternate implementations to support the same API so some modules could work cross implementation - but I suspect that's a non-goal of this PEP :).
Indeed :-) I'm also skeptical that this would actually allow cross-implementation modules to happen. The list of functions that an alternate implementation would have to provide is fairly long.
Just in case you're unaware of it; the company I work for has an open source project called Ironclad.
I was unaware indeed; thanks for pointing this out.
IIUC, it's not just an API emulation, but also an ABI emulation.
In particular we have had to address the issue of the GIL and extensions (IronPython has no GIL) and reference counting (which IronPython also doesn't) use.
I think this somewhat strengthens the point I was trying to make: An alternate implementation that tries to be API compatible has to consider so many things that it is questionable whether making Py_INCREF/DECREF functions would be any simplification.
So I just ask: a) Would it help IronClad if it could restrict itself to PEP 384 compatible modules? b) Would further restrictions in the PEP help that cause?
Regards, Martin
-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog
On Tue, May 19, 2009 at 2:09 AM, William Reade
(for example, all(?) PyCxx modules use PyCode_New and PyFrame_New to get nicer tracebacks)
Specifically for this, I think it'd be nice to expose a function to do this directly. I recently added PyCode_NewEmpty (http://svn.python.org/view?view=rev&revision=72487) to go part of the way here. I didn't go farther because I didn't have a big enough picture. If most uses of PyFrame_New are really just to call into Python with a nice traceback, I think it'd be a good idea to add such a function to ceval.h next to PyEval_Call*(). We can only credibly tell people to use only the ABI functions when we have an ABI replacement for the (sane uses of) non-ABI calls.
William Reade wrote:
2) Since it hasn't always been in place, its introduction won't help me in the short term: there are an awful lot of extension modules that use excluded functions (for example, all(?) PyCxx modules use PyCode_New and PyFrame_New to get nicer tracebacks), and I'll still have to handle all these cases until everyone is up-to-date with whatever version of Python this gets accepted into. It seems that where I should have said Pyrex, I actually said PyCxx. Sorry for the confusion. Thanks to Barry Scott for pointing it out.
participants (2)
-
Jeffrey Yasskin
-
William Reade