[Python-Dev] [Fwd: Re: PEP 384: Defining a Stable ABI]

[William Reade]

My perspective is as follows:

1) If PEP-384 had always been in place, my life would now be a lot easier.

2) Since it hasn't always been in place, its introduction won't help me 
in the short term: there are an awful lot of extension modules that use 
excluded functions (for example, all(?) PyCxx modules use PyCode_New and 
PyFrame_New to get nicer tracebacks), and I'll still have to handle all 
these cases until everyone is up-to-date with whatever version of Python 
this gets accepted into.

3) Regardless, this PEP makes me very happy, because I can now look 
forward to the glorious day when all extension modules are 
384-compatible (and even *some* modules becoming compatible will make me 
pretty happy).

However, I'm not sure exactly how we can get there from here; I suspect 
that certain features of certain extensions already depend critically 
upon implementation details which will become hidden. The most extreme 
illustrative example I know is from NumPy (in scalarmathmodule.c), and 
looks like this:

            PyInt_Type.tp_as_number = PyLongArrType_Type.tp_as_number;
            PyInt_Type.tp_compare = PyLongArrType_Type.tp_compare;
            PyInt_Type.tp_richcompare = PyLongArrType_Type.tp_richcompare;

...and I fear that many many similar (if perhaps less frightening) 
dependencies exist elsewhere.

Regardless, in answer to the two specific questions you ask:

a) We don't really have that option. However, I would have a much higher 
degree of confidence in running PEP-384-compatible modules under 
Ironclad than I do with current modules, simply because I would no 
longer need to worry about (say) edge cases in which extension writers 
suddenly try to directly access op->ob_type->tp_as_number->nb_power.

b) I can't think of any more useful restrictions. The PEP would solve my 
biggest current worry, which is that my current implementation allows 
managed/unmanaged lists to fall out of sync in certain circumstances 
(but if every list mutation happened via an API call, it wouldn't be an 
issue).

Best Regards
William


Michael Foord wrote:
> The questions from Martin v. Lowis are in the email below.
>
> The PEP under discussion is:
>
> http://www.python.org/dev/peps/pep-0384/
>
> I can proxy any replies you want to send, or you can join Python-dev.
>
> All the best,
>
> Michael
>
> -------- Original Message --------
> Subject: 	Re: [Python-Dev] PEP 384: Defining a Stable ABI
> Date: 	Mon, 18 May 2009 08:00:57 +0200
> From: 	"Martin v. Löwis" <martin at v.loewis.de>
> To: 	Michael Foord <fuzzyman at voidspace.org.uk>
> CC: 	Dino Viehland <dinov at microsoft.com>, Python-Dev 
> <python-dev at python.org>, Unladen Swallow 
> <unladen-swallow at googlegroups.com>, Python List <python-list at python.org>
> References: 	<4A107988.3020202 at v.loewis.de> 
> <ea2499da0905171447n5a353b61w1c98a91bf3617e5c at mail.gmail.com> 
> <4A108ABF.9060909 at v.loewis.de> 
> <ea2499da0905171534vca080dci473d2ee9f9915668 at mail.gmail.com> 
> <1A472770E042064698CB5ADC83A12ACD016E82D6 at TK5EX14MBXC116.redmond.corp.microsoft.com> 
> <4A1097F5.2050009 at v.loewis.de> <4A10A368.1060406 at voidspace.org.uk>
>
>
>
> >>> It also might make it easier for alternate implementations to support
> >>> the same API so some modules could work cross implementation - but I
> >>> suspect that's a non-goal of this PEP :).
> >>>     
> >>
> >> Indeed :-) I'm also skeptical that this would actually allow
> >> cross-implementation modules to happen. The list of functions that
> >> an alternate implementation would have to provide is fairly long.
> >>
> >>   
> > 
> > Just in case you're unaware of it; the company I work for has an open
> > source project called Ironclad.
>
> I was unaware indeed; thanks for pointing this out.
>
> IIUC, it's not just an API emulation, but also an ABI emulation.
>
> > In particular we have had to address the issue of the GIL and extensions
> > (IronPython has no GIL) and reference counting (which IronPython also
> > doesn't) use.
>
> I think this somewhat strengthens the point I was trying to make: An
> alternate implementation that tries to be API compatible has to consider
> so many things that it is questionable whether making Py_INCREF/DECREF
> functions would be any simplification.
>
> So I just ask:
> a) Would it help IronClad if it could restrict itself to PEP 384
>    compatible modules?
> b) Would further restrictions in the PEP help that cause?
>
> Regards,
> Martin
>   
>
> -- 
> http://www.ironpythoninaction.com/
> http://www.voidspace.org.uk/blog
>
>