-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote:
At 09:20 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
Just to illustrate what I'm trying to achieve. I've updated the Fedora Packaging Guidelines[1]_ to allow two versions of a package to coexist. I'll list here the sqlalchemy-0.4 and -0.3 build steps, filelists, and the output of the test-sql.sh script using this procedure. The end result is what we want but the build step to get there seem a tad fragile and kludgey. Since these are going to become guidelines for all of our python packages, I'd like to know if either: 1) there's a better way to do this or 2) the results I'm achieving are not expected and could disappear with a random upgrade of setuptools.
.. _[1]: http://fedoraproject.org/wiki/PackagingDrafts/PythonEggs
Here's the thing: if you want multiple installed versions of a package, *and* you want there to be a default version, then you *have* to have easy_install (or zc.buildout, or some similar tool) generate the startup scripts for anything that wants to use a non-default version.
You'll have to explain why that is because all my experiments show that to be false. Installing one package via easy_install -m and another via - --single-version-externally-managed gets us 75% of the way to this working. Using easy_install -m for both and then copying/symlinking gets us 100% of the way.
This is true irrespective of the formats of the versions involved, whether they're zipfiles, directories, single-version, or whatever. It's just the nature of the beast, due to the fact that there is a global 'working_set' that lists the projects that are currently on sys.path, and it is initialized when pkg_resources is imported. (And currently, there is no way to *remove* a distribution from the working set.)
Since the working_set doesn't explicitly list eggs unless they are specified in a project's requires.txt it seems like pkg_resources.require() can override a module which is the default because it is installed in site-packages. In fact, I understood this to be a feature of setuptools: allowing someone to override the vendor installed packages in site-packages with your own eggs.
Thus, if you want multiple versions *and* want to be able to select a version after pkg_resources has been imported, you *cannot* have a default version. In other words, the egg must not be on sys.path when pkg_resources is imported. Then, pkg_resources can locate the desired version and add it.
Not quite. In the single-version case, the egg is on sys.path because the module directory is in site-packages. Therefore pkg_resources make a different version importable by placing a different egg's path before site-packages in sys.path. The goal is to have all of these things work: 1) import MODULE should import whatever the vendor decided should be the default. 2) requires.txt in a project's egginfo should work to select a specific version from the installed eggs. 3) In a simple script (without requires.txt), pkg_resources.requires() should work to select a specific version from the installed eggs. I think the basic way to enable this is: For 1) Either install a module that is not an egg into site-packages or install an egg as single-version-externally-managed into site-packages. When the user does import MODULE, python looks in site-packages, finds the MODULE, and imports it without touching any setuptools code. For 2) All installed modules that we want to be selectable must be eggs. This needs to work with both single-version and multiple-version eggs when determining the best version match then, if necessary, modify sys.path to place the best match in front of the other eggs. For 3) No eggs for this module can be on sys.path before the pkg_resources.require() call. This does not count modules brought in via site-packages as those are going to be overridden when we place egg paths before site-packages. This seems to mostly work right now with the procedure I outlined in my previous message. I have to fix the bug found in my other message to see if I can get it to work all the time (and perhaps eliminate the kludginess in my original procedures.) You seem to think there's something wrong with this so there's obviously something you're seeing that I don't. Can you give an example of where this will fail? - -Toshio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFG2dUKX6yAic2E7kgRAiMnAJ40guBFKrh0V3Yl+kzJqRlZ6V5TJgCgrA/j fnHPHB9Q6533xSAzELAV6Dc= =swaR -----END PGP SIGNATURE-----