[Python-Dev] versioned .so files for Python 3.2

[Matthias Klose]

On 15.07.2010 01:59, Barry Warsaw wrote:
> PEP 384 describes a change to ``PyModule_Create()`` where ``3`` is
> passed as the API version if the extension was complied with
> ``Py_LIMITED_API``.  This should be formalized into an official macro
> called ``PYTHON_ABI_VERSION`` to mirror ``PYTHON_API_VERSION``.  If
> and when the ABI changes in an incompatible way, this version number
> would be bumped.  To facilitate sharing, Python would be extended to
> search for extension modules with the ``PYTHON_ABI_VERSION`` number in
> its name.  The prefix ``abi`` is reserved for Python's use.
>
> Thus for example, an initial implementation of PEP 384, compiled with
> `--with-so-abi-tag=cpython-xy` would search for the following file
> names when extension module `foo` is imported (in this order)::
>
>      foo.abi3.so
>      foo.cpython-xy.so
>      foo.so
>
> The distutils [7]_ ``build_ext`` command would also have to be
> extended to compile to shared library files with the ``abi3`` tag,
> when the module author indicates that their extension supports that
> version of the ABI.  This could be done in a backward compatible way
> by adding a keyword argument to the ``Extension`` class, such as::
>
>      Extension('foo', ['foo.c'], abi=3)

I like the proposal, but IMO it is too unspecific about the abi tag.  Assume 
that an extension is built with such a configured python and then tried to run 
with an abi compatible configured python, but with a slightly different version 
tag, the extension won't be found.  Differing file names per configuration 
should be avoided. Proposing

   1) Remove the configure option and use the new naming using the tag for
      all configurations unconditionally.  Everybody can expect the same
      name on every installation, and PEP 384 most likely will require
      using a tag too.
      As Amaury did point out, there is a use case in that PyPy can use this
      tag too to build extensions only usable by PyPy.

   2) As PEP 3147 defines a non-configurable name for .pyc files, this PEP
      should define a non-configurable way for the tag. The tag should
      include all information which currently makes an extension ABI
      incompatible:
       - the python implementation (cpython, PyPy, ...)
       - the python version (3,2, 3.3, ...)
       - unicode configure option (--with-wide-unicode, 16 or 32)
       - platform information (necessary?)
      If this list changes for coming python versions, then it can be
      extended.
      Barry pointed out on irc chat that people might want to build
      experimental ABI incompatible versions, which should have its
      own tag. If this is wanted, please provide a configure option
      which lets extend/append to the tag.

   3) In case that 1) is not acceptable, the --with-so-abi-tag option
      should be implemented in such a way that it isn't required to
      take any arguments, and in this case to default to the fixed
      naming schema described in 2).

Matthias