On 25.06.2010 22:12, James Y Knight wrote:
On Jun 25, 2010, at 4:53 AM, Scott Dial wrote:
On 6/24/2010 8:23 PM, James Y Knight wrote:
On Jun 24, 2010, at 5:53 PM, Scott Dial wrote:
If the package has .so files that aren't compatible with other version of python, then what is the motivation for placing that in a shared location (since it can't actually be shared)
Because python looks for .so files in the same place it looks for the .py files of the same package.
My suggestion was that a package that contains .so files should not be shared (e.g., the entire lxml package should be placed in a version-specific path). The motivation for this PEP was to simplify the installation python packages for distros; it was not to reduce the number of .py files on the disk.
Placing .so files together does not simplify that install process in any way. You will still have to handle such packages in a special way.
This is a good point, but I think still falls short of a solution. For a package like lxml, indeed you are correct. Since debian needs to build it once per version, it could just put the entire package (.py files and .so files) into a different per-python-version directory.
This is what is currently done. This will increase the size of packages by duplicating the .py files, or you have to install the .py in a common location (irrelevant to sys.path), and provide (sym)links to the expected location. A "different per-python-version directory" also has the disadvantage that file conflicts between (distribution) packages cannot be detected.
However, then you have to also consider python packages made up of multiple distro packages -- like twisted or zope. Twisted includes some C extensions in the core package. But then there are other twisted modules (installed under a "twisted.foo" name) which do not include C extensions. If the base twisted package is installed under a version-specific directory, then all of the submodule packages need to also be installed under the same version-specific directory (and thus built for all versions).
In the past, it has proven somewhat tricky to coordinate which directory the modules for package "foo" should be installed in, because you need to know whether *any* of the related packages includes a native ".so" file, not just the current package.
The converse situation, where a base package did *not* get installed into a version-specific directory because it includes no native code, but a submodule *does* include a ".so" file, is even trickier.
I don't think that installation into different locations based on the presence of extension will work. Should a location really change if an extension is added as an optimization? Splitting a (python) package into different installation locations should be avoided. Matthias