Bug in setuptools version parsing when appending '.dev'?

Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1') True pv('1.0a1') < pv('1.0') True pv('1.0a1.dev') < pv('1.0.dev') False pv('1.0a1') < pv('1.0.dev') False import setuptools setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'. If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing. -- Dave

On Wed, Jul 30, 2008 at 8:05 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1') True pv('1.0a1') < pv('1.0') True pv('1.0a1.dev') < pv('1.0.dev') False pv('1.0a1') < pv('1.0.dev') False import setuptools setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
This looks wrong to me too. One would think that the development version 3 (final) would be newer than any 3.beta version. But both dev and b are pre-release tags with dev being explicitly mapped to '@' in the parsed version so that it is treated as younger than beta: In [2]: v('3.0.dev') Out[2]: ('00000003', '*@', '*final') In [3]: v('3.0b1') Out[3]: ('00000003', '*b', '00000001', '*final') The rationale is probably that dev releases represent "early" development releases, while beta releases are closer to the final release than dev releases. What I think is missing is a way of saying (silently) that this a dev of the final, and that final is newer than beta (and especially a dev of a beta).
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
To work with what we have now, I think you need to set the trunk version to 3b2.dev (the next in-development version). And then when you're ready for the final version of 3, bump your working copy of the trunk to 3 (final) and do a test build with the .dev dropped (perhaps from the command-line with python setup.py egg_info -RDb "" sdist bdist_egg) in a virtualenv or similar sandbox where you can easily manually wipe site-packages in case you need to iterate. Once that's good, commit, branch and tag, release, and then immediately commit the trunk version to 3.1.dev (3.0.1 bug-fix releases will be prepared in the branch).

Alexander Michael wrote:
On Wed, Jul 30, 2008 at 8:05 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1')
True
pv('1.0a1') < pv('1.0')
True
pv('1.0a1.dev') < pv('1.0.dev')
False
pv('1.0a1') < pv('1.0.dev')
False
import setuptools setuptools.__version__
'0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
This looks wrong to me too. One would think that the development version 3 (final) would be newer than any 3.beta version. But both dev and b are pre-release tags with dev being explicitly mapped to '@' in the parsed version so that it is treated as younger than beta:
In [2]: v('3.0.dev') Out[2]: ('00000003', '*@', '*final')
In [3]: v('3.0b1') Out[3]: ('00000003', '*b', '00000001', '*final')
The rationale is probably that dev releases represent "early" development releases, while beta releases are closer to the final release than dev releases. What I think is missing is a way of saying (silently) that this a dev of the final, and that final is newer than beta (and especially a dev of a beta).
Actually, you can append the tag 'final' and it seems to work fine:
pv('3.0.final.dev') > pv(3.0b1') True
I just hate to add the tag 'final' to a final release due to the extra typing it causes people. It seems to me the community expectation is to be able to do things like "easy_install Traits==3.0" rather than "easy_install Traits==3.0.final"
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
To work with what we have now, I think you need to set the trunk version to 3b2.dev (the next in-development version). And then when you're ready for the final version of 3, bump your working copy of the trunk to 3 (final) and do a test build with the .dev dropped (perhaps from the command-line with python setup.py egg_info -RDb "" sdist bdist_egg) in a virtualenv or similar sandbox where you can easily manually wipe site-packages in case you need to iterate. Once that's good, commit, branch and tag, release, and then immediately commit the trunk version to 3.1.dev (3.0.1 bug-fix releases will be prepared in the branch).
The problem here is that we *released* Traits 3.0. It is no longer a beta. But other projects that wanted to rely on it as soon as it was in a stable beta put in a requirement of "Traits >=3.0b1". Unfortunately, if developers check-out the tag of the release and do a build, they get Traits with version "3.0.dev-rXXXXX' which setuptools says doesn't satisfy "Traits >=3.0b1". They get the '.dev' because we have historically not modified the source's setup.cfg when tagging the release (we do NO changes between the last build/test cycle and the tagging.) -- Dave

On Mon, Aug 4, 2008 at 1:24 PM, Dave Peterson <dpeterson@enthought.com> wrote:
Alexander Michael wrote:
The rationale is probably that dev releases represent "early" development releases, while beta releases are closer to the final release than dev releases. What I think is missing is a way of saying (silently) that this a dev of the final, and that final is newer than beta (and especially a dev of a beta).
Actually, you can append the tag 'final' and it seems to work fine:
pv('3.0.final.dev') > pv(3.0b1') True
I just hate to add the tag 'final' to a final release due to the extra typing it causes people. It seems to me the community expectation is to be able to do things like "easy_install Traits==3.0" rather than "easy_install Traits==3.0.final"
I thought this might work, which is why I suggested that maybe it could be done "silently". For example, in the setup.py files but not at the easy_install command-line. This implicit complication would probably lead to other wonkiness though...
To work with what we have now, I think you need to set the trunk version to 3b2.dev (the next in-development version). And then when you're ready for the final version of 3, bump your working copy of the trunk to 3 (final) and do a test build with the .dev dropped (perhaps from the command-line with python setup.py egg_info -RDb "" sdist bdist_egg) in a virtualenv or similar sandbox where you can easily manually wipe site-packages in case you need to iterate. Once that's good, commit, branch and tag, release, and then immediately commit the trunk version to 3.1.dev (3.0.1 bug-fix releases will be prepared in the branch).
=3.0b1". They get the '.dev' because we have historically not modified
The problem here is that we *released* Traits 3.0. It is no longer a beta. But other projects that wanted to rely on it as soon as it was in a stable beta put in a requirement of "Traits >=3.0b1". Unfortunately, if developers check-out the tag of the release and do a build, they get Traits with version "3.0.dev-rXXXXX' which setuptools says doesn't satisfy "Traits the source's setup.cfg when tagging the release (we do NO changes between the last build/test cycle and the tagging.)
Again, attempting to offer up practical solutions. Edit the setup.cfg's to drop the dev option in the release branches and update the trunk to the next version (i.e. 3.1.dev-rXXXXX)? That way, checkouts of the release branches will be 3.0-rXXXXX (a post release of 3) and the trunk will be a post of a pre-release that is newer than anything else in the repository. Just a thought... Regards, Alex P.S. And serious congrats on the Traits release-- Traits rocks!

At 07:05 PM 7/30/2008 -0500, Dave Peterson wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1') True pv('1.0a1') < pv('1.0') True pv('1.0a1.dev') < pv('1.0.dev') False pv('1.0a1') < pv('1.0.dev') False import setuptools setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
This is what 'rc' tags are for, actually. You can put your version in the source, and use the -b option to egg_info while doing builds and testing to tack on an 'rc' tag, possibly with a subversion number as well.

Phillip J. Eby wrote:
At 07:05 PM 7/30/2008 -0500, Dave Peterson wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1') True pv('1.0a1') < pv('1.0') True pv('1.0a1.dev') < pv('1.0.dev') False pv('1.0a1') < pv('1.0.dev') False import setuptools setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
This is what 'rc' tags are for, actually. You can put your version in the source, and use the -b option to egg_info while doing builds and testing to tack on an 'rc' tag, possibly with a subversion number as well.
Perhaps I'm missing something, but that doesn't seem like it scales to a community -- not everyone is going to remember to use a '-b' option when building and that is going to cause endless support problems. We do put the following in our setup.cfg so that people don't mistakenly build release versions when not intending to do so: [egg_info] tag_build = .dev tag_svn_revision = 1 Are you staying the standard process for setuptools is to delete / modify this when tagging a release in the repo? If not, how do you avoid the problem of someone building from a source checkout and finding out that 3.0.dev-r1234 doesn't satisfy the '>= 3.0b1' dependency spec? -- Dave

At 12:02 PM 8/4/2008 -0500, Dave Peterson wrote:
Phillip J. Eby wrote:
At 07:05 PM 7/30/2008 -0500, Dave Peterson wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
from pkg_resources import parse_requirement as pv pv('1.0a1.dev') < pv('1.0a1') True pv('1.0a1') < pv('1.0') True pv('1.0a1.dev') < pv('1.0.dev') False pv('1.0a1') < pv('1.0.dev') False import setuptools setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
This is what 'rc' tags are for, actually. You can put your version in the source, and use the -b option to egg_info while doing builds and testing to tack on an 'rc' tag, possibly with a subversion number as well.
Perhaps I'm missing something, but that doesn't seem like it scales to a community -- not everyone is going to remember to use a '-b' option when building and that is going to cause endless support problems.
Only the creator of a release uses that; the community should never be building "release" versions.
We do put the following in our setup.cfg so that people don't mistakenly build release versions when not intending to do so:
[egg_info] tag_build = .dev tag_svn_revision = 1
Are you staying the standard process for setuptools is to delete / modify this when tagging a release in the repo?
I personally don't tag releases in the repository; a release is not a tag. I have an alias called 'release' that maps to 'egg_info -Rdb ""', and I use it when issuing release builds. However, there's nothing stopping you from creating a utility that copies the trunk to a tag, checks out the tag, removes the options, and checks in the tag, if that would be your preferred approach.
If not, how do you avoid the problem of someone building from a source checkout and finding out that 3.0.dev-r1234 doesn't satisfy the '>= 3.0b1' dependency spec?
I bump the version number in SVN immediately after making a release, so that the trunk would be '3.1.dev-r1234', not '3.0dev'. If you're using .dev tags in your egg-info setup, you should bump the version in setup.py immediately *after* releasing, to avoid just this situation.

Phillip J. Eby wrote:
At 12:02 PM 8/4/2008 -0500, Dave Peterson wrote:
Phillip J. Eby wrote:
At 07:05 PM 7/30/2008 -0500, Dave Peterson wrote:
Am I missing something or is the following a bug whereby adding the '.dev' tag is doing something weird?
> from pkg_resources import parse_requirement as pv > pv('1.0a1.dev') < pv('1.0a1') True > pv('1.0a1') < pv('1.0') True > pv('1.0a1.dev') < pv('1.0.dev') False > pv('1.0a1') < pv('1.0.dev') False > import setuptools > setuptools.__version__ '0.6c8'
This is mainly causing us problems when projects try to track alpha and beta level bumps in dependencies, such as when project Foo requires project Bar version 3.0b1 via a requirement like 'Bar >= 3.0b1' (which means we don't want the development prereleases of Bar's first beta release, but anything after that should be okay.) But then when we actually want to release Bar 3.0 and change the version number to just '3.0', suddenly installs fail while we try to run the last set of tests because '3.0.dev' is older than '3.0b1'.
If it is not a bug, how do you handle situations where you want to run that last round of testing prior to tagging and building releases? I'd rather do that AFTER making all source changes, and not have to change the version number after the testing.
This is what 'rc' tags are for, actually. You can put your version in the source, and use the -b option to egg_info while doing builds and testing to tack on an 'rc' tag, possibly with a subversion number as well.
Perhaps I'm missing something, but that doesn't seem like it scales to a community -- not everyone is going to remember to use a '-b' option when building and that is going to cause endless support problems.
Only the creator of a release uses that; the community should never be building "release" versions.
Right, and that's the exact problem! They're grabbing the source via svn checkout from the release tag and doing their own build which ends up getting labeled as '3.0.dev-r1234' (which seems to be exactly the right thing to say to me -- it is an svn build of the 3.0 release) but this does not satisfy dependencies that are in other projects that were put out requiring "Traits >= 3.0b1". Sure I can tell people not to do that when they say it doesn't work, but I think it should work because the pattern works everywhere else (see below.)
We do put the following in our setup.cfg so that people don't mistakenly build release versions when not intending to do so:
[egg_info] tag_build = .dev tag_svn_revision = 1
Are you staying the standard process for setuptools is to delete / modify this when tagging a release in the repo?
I personally don't tag releases in the repository; a release is not a tag. I have an alias called 'release' that maps to 'egg_info -Rdb ""', and I use it when issuing release builds.
Great to hear about what you do, but we frequently have to support customers who need minor updates and having a tag that makes it easy to branch from, or re-create, a release is a very useful thing. We have to many developers who need to restart at a release point to be able to track the code state at release points using any other method we've seen.
However, there's nothing stopping you from creating a utility that copies the trunk to a tag, checks out the tag, removes the options, and checks in the tag, if that would be your preferred approach.
Hmm, and I thought I was clear saying that this is exactly what I do not want to do. :-) Ideally, we don't want to modify tags in anyway. Like you pointed out above, I don't want people accidentally building 'release' versions except the project maintainers.
If not, how do you avoid the problem of someone building from a source checkout and finding out that 3.0.dev-r1234 doesn't satisfy the '>= 3.0b1' dependency spec?
I bump the version number in SVN immediately after making a release, so that the trunk would be '3.1.dev-r1234', not '3.0dev'. If you're using .dev tags in your egg-info setup, you should bump the version in setup.py immediately *after* releasing, to avoid just this situation.
We DO do that. That doesn't solve the problem I'm talking about though. Let me try one more time.... There exist people who prefer to build from source they've gotten from the svn repo. These people don't want to worry about bugs in the trunk / development version so they go grab the source from the latest release tag. They then do a "setup.py bdist_egg" to build an egg that they plan to distribute within their company, work group, what have you as part of their shared project where they defined dependencies as ">=3.0b1" While the latest tagged version is 3.0b1, 3.0b2, 3.0c1, 3.0c2, etc. this process works great as all of those are correctly newer or equal to 3.0b1 even when .dev tags are appended to the version number. BUT, as soon as we tag and release the final 3.0 release, it breaks because 3.0.dev < 3.0b1. It will work fine again when we tag and release 3.1<anything> or 4.0<anything>. It is only the final release of 3.0 that exhibits this problem. This seems like a bug. -- Dave
participants (3)
-
Alexander Michael
-
Dave Peterson
-
Phillip J. Eby