On Jun 30, 2010, at 10:35 PM, M.-A. Lemburg wrote:
The Python default installation dir for libs (including site-packages) is $prefix/lib/pythonX.X, so you already have separate and properly versioned directory paths.
What difference would the extra version on the .so file make in such a setup ?
So on Debian (inherted by Ubuntu), that would be /usr/lib/pythonX.Y and in fact if you look there, you'll see the normal standard library layout you'd expect. There is no site-packages under there though because we install add-ons to dist-packages[*] but functionally it's what you'd expect. However, if you look inside dist-packages, you'll see something a little weird. All the .py files .egg-info files will actually be symlinks into a parallel hierarchy under /usr/share/pyshared. Also, under dist-packages you'll see a directory layout that looks like it normally would under a standard site-packages layout, except that again, all the .py files are symlinked to the pyshared location. The .so and .pyc files in dist-packages will be the actual .so and .pyc files. Why is that? Well, it's because the .py files can be shared but the .pyc and .so files can't. Getting rid of these crufty symlink farms was the initial motivation for PEP 3147. In a PEP 3147 world, pure-Python packages don't need the directory /usr/lib/pythonX.Y/dist-packages. We can just install packages to /usr/share/pyshared and none of the .pyc files will collide between Python versions. It makes distro packaging simpler (no symlinks to keep track of or post-install linking and post-removal clean up to do) and smaller (only one .py file for all supported Python versions). All we have to do is a post-install byte-compilation and we're done. When we remove such a package we only *have* to remove the .py source file because PEP 3147 will not load a __pycache__ pyc if the source is missing (though it's easy enough to clean up the pyc files too). Throw extension modules into the mix and we have the file collision problem again. We can't install _foo.so into /usr/share/pyshared because that's going to be Python ABI-specific. If we can ABI-tag the .so and ensure it will get imported, then we're back to the simpler layout where /usr/share/pyshared is the destination for all installed Python packages. -Barry [*] That's done as a compromise between Debian's interpretation of the FHS and the conflict of that interpretation to from-source installs of Python. Specifically, Debian puts /usr/local/lib/pythonX.Y/dist-packages on the *system* Python's path because that's where it expects system administrators to put their own add-ons to the system Python. That used to be /usr/local/lib/pythonX.Y/site-packages but then if you did a from-source altinstall for example of the same Python version, it would be possible for a "/usr/local/bin/pythonX.Y setup.py install" of a third-party package to break your system Python. Not good! (And yes, it happened to me :).