use of '_' in package name causing version parsing issue?

Today I was informed of an issue in which buildout (with the latest setuptools) is not resolving version numbers properly, causing the wrong package to be selected in some cases. The cause identified was having '_' in the package name. I am still investigating this issue, but thought it would be worth mentioning on the list in case anyone had a quick answer. Here is the note I received which raised the issue: | > The problem was, they used "_" in the package | > name. Buildout and many other systems like setuptools,RPM,DEB etc. | > treat "_" as a special character to distinguish package name and | > version. | > So, it's not a good idea to use "_" in the name of a package. | > The same is the situation with "-" (minus/hyphen) character. | > | > The lesson learned: Don't use "-" and "_" in package name. | | >>> import pkg_resources | >>> a="jiva_interface-2.3.6-py2.6.egg" | >>> b="jiva_interface-2.3.8-py2.6.egg" | >>> pkg_resources.parse_version(a) | ('*jiva', '*_', '*interface', '*final-', '00000002', '00000003', | '00000006', '*final-', '*py', '00000002', '00000006', '*egg', | '*final') | >>> pkg_resources.parse_version(b) | ('*jiva', '*_', '*interface', '*final-', '00000002', '00000003', | '00000008', '*final-', '*py', '00000002', '00000006', '*egg', | '*final') | >>> c="ZeStorage-0.1.1-py2.4.egg" | >>> d="ZeStorage-0.1.2-py2.4.egg" | >>> pkg_resources.parse_version(c) | ('*zestorage', '*final-', '00000000', '00000001', '00000001', | '*final-', '*py', '00000002', '00000004', '*egg', '*final') | >>> pkg_resources.parse_version(d) | ('*zestorage', '*final-', '00000000', '00000001', '00000002', | '*final-', '*py', '00000002', '00000004', '*egg', '*final') |

On Tue, Mar 9, 2010 at 4:03 PM, Brad Allen <bradallen137@gmail.com> wrote:
Today I was informed of an issue in which buildout (with the latest setuptools) is not resolving version numbers properly, causing the wrong package to be selected in some cases. The cause identified was having '_' in the package name. I am still investigating this issue, but thought it would be worth mentioning on the list in case anyone had a quick answer.
Here is the note I received which raised the issue:
Sorry, I didn't realize that this thread was about buildout. Brad, could you send me a simple buildout.cfg (ideally one that installs one project) that illustrates the problem and I'll dig into it. Jim -- Jim Fulton

Hi Jim, On Sat, Mar 13, 2010 at 1:38 AM, Jim Fulton <jim@zope.com> wrote:
On Tue, Mar 9, 2010 at 4:03 PM, Brad Allen <bradallen137@gmail.com> wrote:
Today I was informed of an issue in which buildout (with the latest setuptools) is not resolving version numbers properly, causing the wrong package to be selected in some cases. The cause identified was having '_' in the package name. I am still investigating this issue, but thought it would be worth mentioning on the list in case anyone had a quick answer.
Here is the note I received which raised the issue:
Sorry, I didn't realize that this thread was about buildout.
Brad, could you send me a simple buildout.cfg (ideally one that installs one project) that illustrates the problem and I'll dig into it.
I have created a branch to demonstrate the issue: svn+ssh://baijum@svn.zope.org/repos/main/zc.buildout/branches/baijum-project-name-with-underscore Please look at this test file: src/zc/buildout/project_name_issue.txt http://svn.zope.org/repos/main/zc.buildout/branches/baijum-project-name-with... When I run the test cases like this, I am getting error: $ ./bin/test -t project_name_issue Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Failure in test /mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt Failed doctest test for project_name_issue.txt File "/mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt", line 0 ---------------------------------------------------------------------- File "/mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt", line 45, in project_name_issue.txt Failed example: print system(buildout), Expected: Getting distribution for 'foo-bar'. Got foo-bar 1. Installing foo_bar. recipe v1 Got: Getting distribution for 'foo-bar'. Got foo-bar 1. While: Installing. Getting section foo_bar. Initializing section foo_bar. Installing recipe foo_bar. Getting distribution for 'foo-bar'. Error: Picked: foo-bar = V.V Ran 4 tests with 1 failures and 0 errors in 0.720 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Regards, Baiju M

On Mon, Mar 15, 2010 at 2:11 PM, Baiju M <mbaiju@zeomega.com> wrote:
Hi Jim,
On Sat, Mar 13, 2010 at 1:38 AM, Jim Fulton <jim@zope.com> wrote:
On Tue, Mar 9, 2010 at 4:03 PM, Brad Allen <bradallen137@gmail.com> wrote:
Today I was informed of an issue in which buildout (with the latest setuptools) is not resolving version numbers properly, causing the wrong package to be selected in some cases. The cause identified was having '_' in the package name. I am still investigating this issue, but thought it would be worth mentioning on the list in case anyone had a quick answer.
Here is the note I received which raised the issue:
Sorry, I didn't realize that this thread was about buildout.
Brad, could you send me a simple buildout.cfg (ideally one that installs one project) that illustrates the problem and I'll dig into it.
I have created a branch to demonstrate the issue:
svn+ssh://baijum@svn.zope.org/repos/main/zc.buildout/branches/baijum-project-name-with-underscore
Please look at this test file: src/zc/buildout/project_name_issue.txt
http://svn.zope.org/repos/main/zc.buildout/branches/baijum-project-name-with...
When I run the test cases like this, I am getting error:
$ ./bin/test -t project_name_issue Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
Failure in test /mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt Failed doctest test for project_name_issue.txt File "/mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt", line 0
---------------------------------------------------------------------- File "/mnt/a9/wa/zc.buildout/src/zc/buildout/project_name_issue.txt", line 45, in project_name_issue.txt Failed example: print system(buildout), Expected: Getting distribution for 'foo-bar'. Got foo-bar 1. Installing foo_bar. recipe v1 Got: Getting distribution for 'foo-bar'. Got foo-bar 1. While: Installing. Getting section foo_bar. Initializing section foo_bar. Installing recipe foo_bar. Getting distribution for 'foo-bar'. Error: Picked: foo-bar = V.V
Ran 4 tests with 1 failures and 0 errors in 0.720 seconds. Tearing down left over layers: Tear down zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
I tried this patch and the test is passing: Index: src/zc/buildout/buildout.py =================================================================== --- src/zc/buildout/buildout.py (revision 109967) +++ src/zc/buildout/buildout.py (working copy) @@ -265,7 +265,10 @@ versions = options.get('versions') if versions: - zc.buildout.easy_install.default_versions(dict(self[versions])) + safe_versions = {} + for package, version in self[versions].items(): + safe_versions[pkg_resources.safe_name(package)] = version + zc.buildout.easy_install.default_versions(safe_versions) prefer_final = options.get('prefer-final', 'false') if prefer_final not in ('true', 'false'): -- Baiju M
participants (3)
-
Baiju M
-
Brad Allen
-
Jim Fulton