Disabling --single-version-externally-managed
I have some packages that I'm using the following with: %{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT I now need to install multiple versions of one of the packages and so need to disable the implicit --single-version-externally-managed. How can I do so and still use setup.py install? -- Ignacio Vazquez-Abrams <ivazqueznet@gmail.com>
At 01:14 AM 8/29/2007 -0400, Ignacio Vazquez-Abrams wrote:
I have some packages that I'm using the following with:
%{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
I now need to install multiple versions of one of the packages and so need to disable the implicit --single-version-externally-managed. How can I do so and still use setup.py install?
You can't. You can use something like this, though, to make an RPM that installs a non-active egg: setup.py easy_install --prefix=$RPM_BUILD_ROOT/usr -mx . The -mx means not to install a .pth or scripts, and the --prefix will set the base installation directory. The '.' means install the package whose source is in the current directory.
Phillip J. Eby wrote:
At 01:14 AM 8/29/2007 -0400, Ignacio Vazquez-Abrams wrote:
I have some packages that I'm using the following with:
%{__python} setup.py install -O1 --skip-build --root $RPM_BUILD_ROOT
I now need to install multiple versions of one of the packages and so need to disable the implicit --single-version-externally-managed. How can I do so and still use setup.py install?
You can't. You can use something like this, though, to make an RPM that installs a non-active egg:
setup.py easy_install --prefix=$RPM_BUILD_ROOT/usr -mx .
Hmmm... So I've been experimenting a bit with this to make some new guidelines for actually utilizing setuptools to make rpm packages [1]_ and found something a little strange. I'm using sqlalchemy0.3 and sqlalchemy0.4 as my test case. Install sqlalchemy0.3 using:: easy_install --prefix=$RPM_BUILD_ROOT/usr -m dist/SQLAlchemy-0.3*.egg Then install sqlalchemy0.4 using:: python setup.py install --root=$RPM_BUILD_ROOT%{python_sitelib} This creates the following in site_packages:: SQLAlchemy-0.3.10-py2.5.egg: EGG-INFO sqlalchemy SQLAlchemy-0.4.0beta4-py2.5.egg-info: dependency_links.txt entry_points.txt PKG-INFO SOURCES.txt top_level.txt sqlalchemy (0.4.0) At this point I expect import sqlalchemy to import the 0.4 version and pkg_resources to give me access to 0.3.10 or 0.4.0beta4 depending on how I specify my versions. But in fact, 0.4.0 is not available:: pkg_require >=0.3,<0.4.0beta1... Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 528, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (SQLAlchemy 0.4.0beta4 (/usr/lib/python2.5/site-packages), Requirement.parse('SQLAlchemy>=0.3,<0.4.0beta1')) This also happens when I use *.pth files instead of installing 0.4 as an active egg. Is this a bug? I can get my desired behaviour by installing both 0.3 and 0.4 as inactive eggs. Then manually copying/symlinking the 0.4 version into site-packages. I'll attach a small script I use to test this. .. _[1]: http://fedoraproject.org/wiki/PackagingDrafts/PythonEggs -Toshio
At 03:26 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
I'm using sqlalchemy0.3 and sqlalchemy0.4 as my test case. Install sqlalchemy0.3 using:: easy_install --prefix=$RPM_BUILD_ROOT/usr -m dist/SQLAlchemy-0.3*.egg
Then install sqlalchemy0.4 using:: python setup.py install --root=$RPM_BUILD_ROOT%{python_sitelib}
This creates the following in site_packages:: SQLAlchemy-0.3.10-py2.5.egg: EGG-INFO sqlalchemy SQLAlchemy-0.4.0beta4-py2.5.egg-info: dependency_links.txt entry_points.txt PKG-INFO SOURCES.txt top_level.txt sqlalchemy (0.4.0)
At this point I expect import sqlalchemy to import the 0.4 version and pkg_resources to give me access to 0.3.10 or 0.4.0beta4 depending on how I specify my versions. But in fact, 0.4.0 is not available::
pkg_require >=0.3,<0.4.0beta1... Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 528, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (SQLAlchemy 0.4.0beta4 (/usr/lib/python2.5/site-packages), Requirement.parse('SQLAlchemy>=0.3,<0.4.0beta1'))
This also happens when I use *.pth files instead of installing 0.4 as an active egg.
I'm not sure what you mean by using .pth files here. If you did the easy_install without the '-m', that would make the 0.3 version the active (default) version, though, and it would override the default-active 0.4 version.
Is this a bug?
Not really. Once pkg_resources has been imported, it's too late to request an inactive version that conflicts with an active version of the same project, as the default working set has already been configured by that time. However, if you create a project that depends on the 0.3 SQLAlchemy, and run a script from that project, it will see the correct version. pkg_resources can resolve the version conflict automatically under those circumstances. Anyway, the point here is that a default version is a default version, period. The only way it can be overridden is by a conflicting requirement defined for a script.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote:
At 03:26 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
At this point I expect import sqlalchemy to import the 0.4 version and pkg_resources to give me access to 0.3.10 or 0.4.0beta4 depending on how I specify my versions. But in fact, 0.4.0 is not available::
pkg_require >=0.3,<0.4.0beta1... Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 528, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (SQLAlchemy 0.4.0beta4 (/usr/lib/python2.5/site-packages), Requirement.parse('SQLAlchemy>=0.3,<0.4.0beta1'))
This also happens when I use *.pth files instead of installing 0.4 as an active egg.
I'm not sure what you mean by using .pth files here. If you did the easy_install without the '-m', that would make the 0.3 version the active (default) version, though, and it would override the default-active 0.4 version.
I tried manually creating a .pth file that lists one or the other of the eggs.
Is this a bug?
Not really. Once pkg_resources has been imported, it's too late to request an inactive version that conflicts with an active version of the same project, as the default working set has already been configured by that time.
What determines if a version is active or inactive?
However, if you create a project that depends on the 0.3 SQLAlchemy, and run a script from that project, it will see the correct version. pkg_resources can resolve the version conflict automatically under those circumstances.
If I understand you correctly, this is exactly the situation that is causing the error above. I guess I should have been more explicit in what was being invoked to cause that traceback. It's the second test in my test script: python -c 'import pkg_resources; pkg_resources.require("SQLAlchemy>=0.3,<0.4.0be ta1"); import sqlalchemy; print sqlalchemy.__version__'
Anyway, the point here is that a default version is a default version, period. The only way it can be overridden is by a conflicting requirement defined for a script.
1) Is the default version different from the active version? If so how? 2) Is what I'm doing above with pkg_resources.require() what you mean by specifying a conflicting requirement? 2b) If so, is it a bug that it doesn't work? Thanks, - -Toshio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFG2MawX6yAic2E7kgRAuGgAJ4nEa7ZOk/hZ0a0m89tz8hKDiKH0wCeLCgd rtXcPuxOT5HPrT7bmEHrQRc= =Kxbb -----END PGP SIGNATURE-----
[Resending with gzipped complete filelist as the previous try was too large to send to the list] Just to illustrate what I'm trying to achieve. I've updated the Fedora Packaging Guidelines[1]_ to allow two versions of a package to coexist. I'll list here the sqlalchemy-0.4 and -0.3 build steps, filelists, and the output of the test-sql.sh script using this procedure. The end result is what we want but the build step to get there seem a tad fragile and kludgey. Since these are going to become guidelines for all of our python packages, I'd like to know if either: 1) there's a better way to do this or 2) the results I'm achieving are not expected and could disappear with a random upgrade of setuptools. .. _[1]: http://fedoraproject.org/wiki/PackagingDrafts/PythonEggs Build ----- sqlalchemy-0.3 compat package:: CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py bdist_egg mkdir -p %{python_sitelib} easy_install -m --prefix $RPM_BUILD_ROOT%{_usr} dist/*.egg sqlalchemy-0.4 default package:: CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py bdist_egg mkdir -p %{python_sitelib} easy_install -m --prefix %{_usr} --always-unzip dist/*.egg cd %{python_sitelib}/%{srcname}-%{version}%{betaver}-py%{pyver}.egg mv sqlalchemy .. ln -s ../sqlalchemy . The compat package is pretty straighforward. However, building the default package seems overly complex. It seems like we should be able to do this:: CFLAGS="$RPM_OPT_FLAGS" %{__python} setup.py build %{__python} setup.py install --skip-build --root $RPM_BUILD_ROOT But that yields the tracebacks when using pkg_resource.require() to try to run 0.3. truncated filelist ------------------ Full filelist attached. These are the main toplevel directories to show where the important pieces are. The sqlalchemy directories all contain a version of the python module. (SQLAlchemy-0.4.egg/sqlalchemy is actually a symlink to site-packages/sqlalchemy but that doesn't matter. Those can be reversed or they can be copies with the same results). site-packages/SQLAlchemy-0.3.10-py2.5.egg site-packages/SQLAlchemy-0.3.10-py2.5.egg/EGG-INFO site-packages/SQLAlchemy-0.3.10-py2.5.egg/sqlalchemy site-packages/SQLAlchemy-0.4.0beta4-py2.5.egg site-packages/SQLAlchemy-0.4.0beta4-py2.5.egg/EGG-INFO site-packages/SQLAlchemy-0.4.0beta4-py2.5.egg/sqlalchemy site-packages/sqlalchemy test-sql.sh output ------------------ import sqlalchemy... 0.4.0beta4 pkg_require >=0.3,<0.4.0beta1... 0.3.10 pkg_require... 0.4.0beta4 pkg_require >=0.3... 0.4.0beta4 pkg_require <= 0.4.10... 0.4.0beta4 pkg_require <=0.3.12... 0.3.10 -Toshio
At 06:56 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
Phillip J. Eby wrote:
At 03:26 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
At this point I expect import sqlalchemy to import the 0.4 version and pkg_resources to give me access to 0.3.10 or 0.4.0beta4 depending on how I specify my versions. But in fact, 0.4.0 is not available::
pkg_require >=0.3,<0.4.0beta1... Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 626, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/lib/python2.5/site-packages/pkg_resources.py", line 528, in resolve raise VersionConflict(dist,req) # XXX put more info here pkg_resources.VersionConflict: (SQLAlchemy 0.4.0beta4 (/usr/lib/python2.5/site-packages), Requirement.parse('SQLAlchemy>=0.3,<0.4.0beta1'))
This also happens when I use *.pth files instead of installing 0.4 as an active egg.
I'm not sure what you mean by using .pth files here. If you did the easy_install without the '-m', that would make the 0.3 version the active (default) version, though, and it would override the default-active 0.4 version.
I tried manually creating a .pth file that lists one or the other of the eggs.
That won't work to make the egg the default version. You have to have easy_install generate the .pth file, so that the necessary magic is included. Normally, paths in .pth files are added to the *end* of sys.path, which means the single-version egg will take precedence. easy_install adds special incantations to its .pth files so that the eggs it installs have higher precedence than everything else.
What determines if a version is active or inactive?
Whether it's listed on sys.path. Your single-version egg's location was site-packages, and thus it's always on sys.path.
However, if you create a project that depends on the 0.3 SQLAlchemy, and run a script from that project, it will see the correct version. pkg_resources can resolve the version conflict automatically under those circumstances.
If I understand you correctly, this is exactly the situation that is causing the error above. I guess I should have been more explicit in what was being invoked to cause that traceback. It's the second test in my test script:
python -c 'import pkg_resources; pkg_resources.require("SQLAlchemy>=0.3,<0.4.0be ta1"); import sqlalchemy; print sqlalchemy.__version__'
Please reread what I said above: "if you create a *project* ... and run a script *from that project*". That has nothing to do with what you're trying here, where you neither have a project nor a script generated from that project. (You don't even have a script!) The script wrappers generated by easy_install have code that handles this conflict issue. Your test does not.
Anyway, the point here is that a default version is a default version, period. The only way it can be overridden is by a conflicting requirement defined for a script.
1) Is the default version different from the active version? If so how?
A default version is the version that's active by default. :) If it's listed in a .pth *generated by easy_install*, it will be the default version. Otherwise, if it's a single-version egg on sys.path, it will be the default version. Otherwise, if it's listed in a handmade (non-magic) .pth file, it will be the default version. If it's somehow otherwise put on sys.path, it will be the default version. If none of the above apply, there is no default version.
2) Is what I'm doing above with pkg_resources.require() what you mean by specifying a conflicting requirement?
No.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Phillip J. Eby wrote:
At 06:56 PM 8/31/2007 -0700, Toshio Kuratomi wrote:
I tried manually creating a .pth file that lists one or the other of the eggs.
That won't work to make the egg the default version. You have to have easy_install generate the .pth file, so that the necessary magic is included.
Normally, paths in .pth files are added to the *end* of sys.path, which means the single-version egg will take precedence. easy_install adds special incantations to its .pth files so that the eggs it installs have higher precedence than everything else.
Good to know. It doesn't affect what I tested as I used a manually created .pth file *instead* of using single-version-externally-managed. So the eggs were arranged as: SQLAlchemy-0.4[...].egg/sqlalchemy SQLAlchemy-0.3[...].egg/sqlalchemy sqlalchemy.pth (containing the path into one of the eggs) I'd rather do without .pth's in the Guidelines though, as they seem to duplicate what can already be achieved by installing one egg as single-version. [...] The rest of this is miscommunication based on my using terms incorrectly. I'll reply to the other message with something meaningful now that I understand: active version -- egg on sys.path inactive version -- egg cannot be found by python as it is not on sys.path default version -- version of a module that comes first on sys.path and therefore will be selected from a bare import project -- setuptools managed project that uses requires.txt to manage conflicting versions. If I've still got those wrong, let me know :-) - -Toshio -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFG2b2xX6yAic2E7kgRAg4MAJ9114KrfVWHxBa3MdglZPMUqoOTJACgiDKm 7BG+jEMItLfGplZ2rffDoGQ= =7IK3 -----END PGP SIGNATURE-----
participants (3)
-
Ignacio Vazquez-Abrams -
Phillip J. Eby -
Toshio Kuratomi