[Python-Dev] versioned .so files for Python 3.2

[James Y Knight]

On Jun 25, 2010, at 4:53 AM, Scott Dial wrote:

> On 6/24/2010 8:23 PM, James Y Knight wrote:
>> On Jun 24, 2010, at 5:53 PM, Scott Dial wrote:
>>> If the package has .so files that aren't compatible with other  
>>> version
>>> of python, then what is the motivation for placing that in a shared
>>> location (since it can't actually be shared)
>>
>> Because python looks for .so files in the same place it looks for the
>> .py files of the same package.
>
> My suggestion was that a package that contains .so files should not be
> shared (e.g., the entire lxml package should be placed in a
> version-specific path). The motivation for this PEP was to simplify  
> the
> installation python packages for distros; it was not to reduce the
> number of .py files on the disk.
>
> Placing .so files together does not simplify that install process in  
> any
> way. You will still have to handle such packages in a special way.


This is a good point, but I think still falls short of a solution. For  
a package like lxml, indeed you are correct. Since debian needs to  
build it once per version, it could just put the entire package (.py  
files and .so files) into a different per-python-version directory.

However, then you have to also consider python packages made up of  
multiple distro packages -- like twisted or zope. Twisted includes  
some C extensions in the core package. But then there are other  
twisted modules (installed under a "twisted.foo" name) which do not  
include C extensions. If the base twisted package is installed under a  
version-specific directory, then all of the submodule packages need to  
also be installed under the same version-specific directory (and thus  
built for all versions).

In the past, it has proven somewhat tricky to coordinate which  
directory the modules for package "foo" should be installed in,  
because you need to know whether *any* of the related packages  
includes a native ".so" file, not just the current package.

The converse situation, where a base package did *not* get installed  
into a version-specific directory because it includes no native code,  
but a submodule *does* include a ".so" file, is even trickier.

James