[Python-Dev] versioned .so files for Python 3.2

[Brett Cannon]

On Thu, Jun 24, 2010 at 08:50, Barry Warsaw <barry at python.org> wrote:
> This is a follow up to PEP 3147.  That PEP, already implemented in Python 3.2,
> allows for Python source files from different Python versions to live together
> in the same directory.  It does this by putting a magic tag in the .pyc file
> name and placing the .pyc file in a __pycache__ directory.
>
> Distros such as Debian and Ubuntu will use this to greatly simplifying
> deploying Python, and Python applications and libraries.  Debian and Ubuntu
> usually ship more than one version of Python, and currently have to play
> complex games with symlinks to make this work.  PEP 3147 will go a long way to
> eliminating the need for extra directories and symlinks.
>
> One more thing I've found we need though, is a way to handled shared libraries
> for extension modules.  Just as we can get name collisions on foo.pyc, we can
> get collisions on foo.so.  We obviously cannot install foo.so built for Python
> 3.2 and foo.so built for Python 3.3 in the same location.  So symlink
> nightmare's mini-me is back.
>
> I have a fairly simple fix for this.  I'd actually be surprised if this hasn't
> been discussed before, but teh Googles hasn't turned up anything.
>
> The idea is to put the Python version number in the shared library file name,
> and extend .so lookup to find these extended file names.  So for example, we'd
> see foo.3.2.so instead, and Python would know how to dynload both that and the
> traditional foo.so file too (for backward compatibility).
>
> (On file naming: the original patch used foo.so.3.2 and that works just as
> well, but I thought there might be tools that expect exactly a '.so' suffix,
> so I changed it to put the Major.Minor version number to the left of the
> extension.  The exact naming scheme is of course open to debate.)
>

While the idea is fine with me since I won't have any of my
directories cluttered with multiple .so files, I would still want to
add some moniker showing that the version number represents the
interpreter and not the .so file. If I read "foo.3.2.so", that naively
seems to mean to mean the foo module's 3.2 release is what is in
installed, not that it's built for CPython 3.2. So even though it
might be redundant, I would still want the VM name added.

Adding the VM name also doesn't make extension modules the exclusive
domain of CPython either. If some other VM decides to make their own
.so files that are not binary compatible then we should not preclude
that as this solution it is nothing more than it makes a string
comparison have to look at 7 more characters.

-Brett

P.S.: I wish we could drop use of the 'module.so' variant at the same
time, for consistency sake and to cut out a stat call, but I know that
is asking too much.