[Python-Dev] PEP 384: Defining a Stable ABI
mal at egenix.com
Mon May 25 19:41:54 CEST 2009
Martin v. Löwis wrote:
> Thomas Wouters reminded me of a long-standing idea; I finally
> found the time to write it down.
> Please comment!
Up until this PEP proposal, we had a very simple scheme for
the Python C-API: all documented functions and variables with
a "Py" prefix were part of the C-API, everything else was not
and could change between releases (in particular the private
"_Py" prefix APIs).
Changing the published APIs was considered a bad thing in the
2.x development process and generally required a good reason
to get supported.
Changing private functions or ones that were not documented
was generally never a big problem.
Now, with the PEP, I have a feeling that the Python C-API
will in effect be limited to what's in the PEP's idea of
a usable ABI and open up the non-inluded public C-APIs
to the same rate of change as the private APIs.
If that's the case, the PEP should be discussed on the C-API
list first, in order to identify a complete list of APIs that
is supposed to define the Python C-API. Ideally, all other
APIs would then need to be made private. However, I doubt that
this is possible before switching to Python 4.0.
Then again, I'm not sure whether that's what you're aiming for...
An optional cross-version ABI would certainly be a good thing.
Limiting the Python C-API would be counterproductive.
> During the compilation of applications, the preprocessor macro
> Py_LIMITED_API must be defined. Doing so will hide all definitions
> that are not part of the ABI.
So extensions wanting to use the full Python C-API as documented
in the C-API docs will still be able to do this, right ?
> Type Objects
> The structure of type objects is not available to applications;
> declaration of "static" type objects is not possible anymore
> (for applications using this ABI).
Hmm, that's going to create big problems for extensions that
want to expose a C-API for their types: Type checks are normally
done by pointer comparison using those static type objects.
> Functions and function-like Macros
> Function-like macros (in particular, field access macros) remain
> available to applications, but get replaced by function calls
> (unless their definition only refers to features of the ABI, such
> as the various _Check macros)
Including Py_INCREF()/Py_DECREF() ?
> Excluded Functions
> Functions declared in the following header files are not part
> of the ABI:
> - cellobject.h
> - classobject.h
> - code.h
> - frameobject.h
> - funcobject.h
> - genobject.h
> - pyarena.h
> - pydebug.h
> - symtable.h
> - token.h
> - traceback.h
I don't think that's feasable: you basically remove all introspection
functions that way.
This will need a more fine-grained approach.
> On Windows, applications shall link with python3.dll;
You mean: extensions that were compiled with Py_LIMITED_API, right ?
> an import
> library python3.lib will be available. This DLL will redirect all of
> its API functions through /export linker options to the full
> interpreter DLL, i.e. python3y.dll.
What if you mix extensions that use the full C-API with ones
that restrict themselves to the limited version ?
Would creating a Python object in a full-API extension and
free'ing it in a limited-API extension cause problems ?
> Implementation Strategy
> This PEP will be implemented in a branch, allowing users to check
> whether their modules conform to the ABI. To simplify this testing, an
> additional macro Py_LIMITED_API_WITH_TYPES will expose the existing
> type object layout, to let users postpone rewriting all types. When
> the this branch is merged into the 3.2 code base, this macro will
> be removed.
Now I'm confused again: this sounds a lot like you do want all extension
writers to only use the limited API.
[And in another post]
>> Something I haven't seen explicitly mentioned as yet (in the PEP or the
>> > python-dev list discussion) are the memory management APIs and the FILE*
>> > APIs which can cause the MSVCRT versioning issues on Windows.
>> > Those would either need to be excluded from the stable ABI or else
>> > changed to use opaque pointers.
> Good point. As a separate issue, I would actually like to deprecate,
> then remove these APIs. I had originally hoped that this would happen
> for 3.0 already, alas, nobody worked on it.
> In any case, I have removed them from the ABI now.
How do you expect Python extensions to allocate memory and objects
in a platform independent way without those APIs ?
And as an aside: Which API families are you referring to ? PyMem_Malloc,
PyObject_Malloc, or PyObject_New ?
Professional Python Services directly from the Source (#1, May 25 2009)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
2009-06-29: EuroPython 2009, Birmingham, UK 34 days to go
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
More information about the Python-list