[Python-Dev] versioned .so files for Python 3.2

[Barry Warsaw]

On Jun 26, 2010, at 10:22 PM, Matthias Klose wrote:

>On 24.06.2010 22:46, Barry Warsaw wrote:
>> So, we could say that PEP 384 compliant extension modules would get written
>> without a version specifier.  IOW, we'd treat foo.so as using the ABI.  It
>> would then be up to the Python runtime to throw ImportErrors if in fact we
>> were loading a legacy, non-PEP 384 compliant extension.
>
>Is it realistic to never break the ABI?  I would think of having the ABI
>encoded in the file name as well, and only bump the ABI if it does change.
>With the "versioned .so files" proposal an ABI bump is necessary with every
>python version, with PEP 384 the ABI bump will be decoupled from the python
>version.

You're right that the ABI will break, requiring a bump, and I think you're
right that this means that PEP 384 compliant shared libraries would have to
have a version number in their file name (assuming the versioned .so proposal
is accepted).

The problem is that we would need two version numbers, one for extension
modules that are not PEP 384 complaint (and thus get bumped for every new
Python version), and one for modules that are PEP 384 compliant (and thus only
get bumped once in a while).  The reason is that I think it will always be the
case that we will have PEP 384 compliant and non-compliant extension modules.

Perhaps identifying the underlying problems will lead to a more acceptable
patch for Python.  My patch tries to take a simple (perhaps too simplistic)
solution, and I'm not married to it, but I think the general idea of versioned
.so files is the right one.

1. The file name extensions that Python searches for are hardcoded and
   compiled in.

dyload_shlib.c hard codes the file name pattern that extension modules must
have in order for Python to load them.  They must be <foo>.so or
<foo>module.so.  This gets compiled into Python at build time and there's no
way for a distro (or anyone else who builds Python from source) to extend the
file name patterns without modifying the source code.

2. The extension that distutils writes for shared libraries is dictated by
   build-time options and cannot be overridden.

When you ./configure Python, autoconf figures out what shared library
extension your platform uses.  It substitutes this into a Makefile variable.
That Makefile gets installed into your system with the base Python package and
distutils parses the Makefile looking for this variable.  When distutils calls
your platform compiler, it uses this Makefile variable as the file name
extension to use for your shared library.  You cannot change this or override
it to get distutils to write some other file name extension, well.

Of these two problems, #1 is more serious because we have to modify the Python
source code to hack in addition shared library search suffixes.  #2 can be
worked around by renaming the .so file after the build.  The disadvantage of
this though is that if you're a local packager, you'll have to remember to do
the same thing if you want multiple Python version support, because distutils
won't take care of it for you.

Maybe that's okay, in which case it would still be good to address #1.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20100630/e2595e84/attachment.pgp>