On Wed, Sep 26, 2012 at 4:24 PM, Donald Stufft wrote:

On Wednesday, September 26, 2012 at 4:00 PM, Toshio Kuratomi wrote:

Note that this was an intentional difference with setuptools.

Do you have any idea _why_? It seems from my perspective (of someone who wasn't really paying attention back then) to be a fairly arbitrary difference in both prior art and what I've personally seen as the common usage that has a good chance of causing confusion (as it already has).

My guess--and this is pure speculation as I wasn't following the original discussions either--is that it fell out of the desire to require versions to be lexicographically ordered. So "dev" would *have* to come after "a", "b", and "c" whether we like it or not. But at some point the convention to use ".dev" was proposed, which solves this issue--this puts it before "a" lexicographically and thus makes more sense in general. Although ".dev" made it into PEP 386 it seems the rest of the PEP wasn't changed to reflect the fact that ".dev" now comes first. Again--just speculation. But I don't see any way that the PEP as written makes sense without assuming some kind of editorial oversight. Erik

On Thursday, September 27, 2012 at 9:12 AM, Erik Bray wrote:

My guess--and this is pure speculation as I wasn't following the original discussions either--is that it fell out of the desire to require versions to be lexicographically ordered. So "dev" would *have* to come after "a", "b", and "c" whether we like it or not. But at some point the convention to use ".dev" was proposed, which solves this issue--this puts it before "a" lexicographically and thus makes more sense in general. Although ".dev" made it into PEP 386 it seems the rest of the PEP wasn't changed to reflect the fact that ".dev" now comes first.

Again--just speculation. But I don't see any way that the PEP as written makes sense without assuming some kind of editorial oversight.

If that's the reason (and I wasn't around during the original discussions) then that's already been broken by rc/final (final is given an internal representation of `z` to allow ordering to work. The way the PEP is written .devN is a pre-release of a specific version (a version being a numbered release of either alpha, beta, candidate, or final). While I think pre-releases of a specific version is likely useful in some context, I think that it should be using an identifier more like `pre`. Using the `dev` tag to me violates what a user expects from history. I'm not discounting that there is a reason that makes sense though, hopefully if there is one someone will be able to come forth with it. I personally feel though that the current behavior is unexpected, will cause confusion as long as releases are being ordered by both pkg_resources and PEP386, and goes against common usage of the release tags. My proposal would be to replace the .devN pre-release tag with .preN, and add a `devN` tag (perhaps optional N, with an omitted N being equal to `dev0`. I feel like this will fit into people's expectations better, and make migration easier.

On Thursday, September 27, 2012 at 11:59 AM, Toshio Kuratomi wrote:

setuptools allows any random string into the versions. Certain strings have special meaning. Unknown strings are lexicographically sorted. Rereading this section of the PEP http://www.python.org/dev/peps/pep-0386/#caveats-of-existing-systems

will give you the background on problems with the setuptools concept. For part of the rationale about .dev, the first paragraph of the rationale is important. setuptools allowed too many things to be allowed into versions. Remembering how the specialcases mix with the lexicographic sorting lead to confusion when some project starts working with non-standard version strings. So one of the goals of PEP 386 was to make a small, discrete number of legal version strings.

No arguments here on the need of this goal.

Keeping that in mind, these are the alpha tags that PEP386 allows: a, b, c, rc, .post, .dev

* a, b, c, and rc are the typical alpha beta release candidate tags. We debated whether to allow the single letter versions or the long versions "alpha", "beta", "rc". In the end, which set of tags was used was a bikeshed argument so it was chosen by PEP-author decree. I personally felt that having both "c" and "rc" was counter to the goals of the PEP but we at least agreed that "rc" would intuitively sort after "c" so it wasn't as bad as having both "alpha" and "a" where there would be no intuitively correct sorting. * For .post there was a desire to have a tag that sorted after release. Choice of .post for the bikeshed color was not controversial. * For .dev there was a desire to have a tag that sorted before release. Choice of .dev for the bikeshed was debated but in the end it was a bikeshed and we were all very tired of it. I'm not sure at this point who was championing .dev over .pre or similar or what their rationale was.

Yes I've read this, but sadly all it says about the issue really is "I don't remember".

I would be for renaming .dev to .pre[1]_ but I would be against the rest of this proposal. Having one and only one way to spell things was one of the goals of the pep. Having two post release tags that don't already have a defined meaning leads to confusion about what the ordering should be:

dev (in my proposal, and in the way I've seen it used) isn't a post release tag, it is a separate release at the same level as alpha, beta, rc, final, and it's meaning tends to be "this is the in development version of what will become release X.Y", so foo-1.0dev1 means the first development release of what will become foo-1.0, (just like foo-1.0a1 is the first alpha release of what will become foo-1.0). With the current layout (and with your changes to the proposal) there is no good way to have a development version. With the current layout the best approximation you can get is 1.0a1.dev1 (similarly with your changes it would be 1.0a1.pre1). On the surface 1.0a1.dev1 looks like it might be alright, but my issues with it: 1. It's not intuitive (at least not to me, development snapshots have always come before alphas in my mind) 2. Goes against what is currently being used for either no good reason or a yet to be remembered good reason. 3. Requires you to decide ahead of time if you're going to go from dev to alpha, or straight to a beta/candidate/final. (This is something that most small projects don't have set in stone). 4. It's semantics are different, this is a development release of 1.0a1 as opposed to a development release of the 1.0 version. 5. It's just plain ugly. So to be clear my proposal would be: 1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1 as opposed to the current: 1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1

On Thursday, September 27, 2012 at 2:03 PM, Toshio Kuratomi wrote:

That's your preference, not capability. I'd also say you should use 1.0.post1 rather than 1.0a1pre1.

So how do you encode the very first development release then if you're using post? Can you point me towards somewhere where their conceptual release cycle is Final -> Dev -> Alpha -> Beta -> RC or Alpha -> Beta -> RC -> Final -> Dev How can you release any sort of Alpha/Beta/RC/Final prior to having done development? If Dev is a post release what do you call it before you've had your first Alpha?

...

On the surface 1.0a1.dev1 looks like it might be alright, but my issues with it:

1. It's not intuitive (at least not to me, development snapshots have always come before alphas in my mind)

There's no such consensus. This is both from dealing with upstream versioning as a distro packager and from discussion with other people who worked on the versioning spec. People were very clear that they wanted to be able to stick dev versions in the middle of their attempt to release alphas, betas, rcs, and finals.

You can see this in current, setuptools-driven practice in the snapshots that pje hosts, for example: http://peak.telecommunity.com/snapshots/

...

2. Goes against what is currently being used for either no good reason or a yet to be remembered good reason.

I've given you the reasons above. I'd also point out that with your clarification that you want .dev to be a toplevel alongside 'a', 'b', and 'c' you are also changing how dev is being used for no good reason -- simply to fit your personal use case.

I'm actually reverting it back to the current defacto standard of what setuptools does. However upon further examination I've figured out I believe what's caused the mismatch between expectations here. Essentially pkg_resources does it both ways. given the tags dev, a, b, c and an implicit f, it allows you to "nest" them. So: 0.1dev1 < 0.1a1 < 0.1b1 < 0.1c1 < 0.1 However it also allows you to add modifiers to these "top level" releases such as: 0.1.dev < 0.1a1.dev1 < 0.1.a1 < 0.1.b1 < 0.1c1 < 0.1 This also makes sense to me in my particular use case, and it sounds like it makes sense in yours? My main concern is that shifting 0.1dev1 from sorting before alpha/beta/candidate to sorting after them is going to cause headaches. So perhaps a better proposal would be to allow .dev at any level (as it does currently), but dictate that .dev sorts first at whatever level it occurs at (which is what setuptools does). This fixes my major problem of getting bitten because setuptools and PEP386 treat "1.0dev1" differently while still giving people the ability to do "1.0a1.dev1". The major drawback being that you can't do a "dev/pre" release between an rc and a final, but as pkg_resources doesn't currently allow for that I would think that isn't a huge concern? That might solve the use case for both sides?

(Note: I assume rc1 is also in all of these examples and sorts between c and "final")

Yes essentially, I mean conceptually c and rc should sort the same but practicality beats purity and what not.

-Toshio

On Thursday, September 27, 2012 at 3:45 PM, Toshio Kuratomi wrote:

then that works perfectly fine for me. To put out a snapshot between rc and final, I'd do 0.2rc1.post1 so not having a pre/dev in there doesn't bother me.

However, now that you mention it, ISTR that this is the way that it was originally and the lack of pre/dev in that position did bother another (set of?) the PEP authors enough that it was changed. For me, switching the search order of that particular piece is a bikeshed so I seem to have reallocated the memory for storing the arguments made in favor of shifting it to the current position :-)

So I guess the question is does losing the ability to not have a dev that is sorted between the rc and the final outweigh the legacy concerns of dealing with the number of packages in the wild (which is hard to count since due to other issues very few non final releases make it to PyPI) that expect 0.1dev1 to sort before an alpha. Personally I would want to fall on the side of not breaking the current assumptions since given a particular version string we do not have a good way of determining if it's expecting dev to fall before alpha or expecting it to fall after the rc. I should note that I originally stumbled upon this issue working on trying to convert existing tools to use PEP386 and warn when a version wasn't PEP386 compat and the difference between the way PEP386 handles the ordering and the way pkg_resources handled the differences caused a problem with a different version getting installed before and after my changes that caused me to dig into why.

On Thu, Sep 27, 2012 at 05:00:56PM -0500, Brad Allen wrote:

On Thu, Sep 27, 2012 at 2:45 PM, Toshio Kuratomi wrote:

...

However, now that you mention it, ISTR that this is the way that it was originally and the lack of pre/dev in that position did bother another (set of?) the PEP authors enough that it was changed. For me, switching the search order of that particular piece is a bikeshed so I seem to have reallocated the memory for storing the arguments made in favor of shifting it to the current position :-)

Why is this perceived as a bikeshed issue? It seems like a discussion about substantial functionality which is tied to the semantics of these special tags. For many projects and organizations, terms like 'alpha' and 'beta' and 'dev' have meaning in a release context, and rearranging the order of these concepts has an impact.

dev has no universal defined relation to alpha, beta, etc in a release context. We all seem to be amenable to: 1.0alpha1.dev1 < 1.0alpha1 But people have not agreed whether: 1.0.dev1 < 1.0alpha1 or 1.0alpha1 < 1.0.dev1 Either of these cases can also be functionally replaced with the appropriate .post tag: 1.0.dev1 < 1.0alpha1 ===> 0.9.post1 < 1.0alpha1 1.0alpha1 < 1.0.dev1 ===> 1.0alpha1 < 1.0rc1.post1 (Substitute the last "final" release for 0.9 in the above and substitute whatever version you're snapshoting after for 1.0rc1) Some organizations and projects use it one way and others the other way. And unlike alpha, beta, rc ordering which is both long established in the computing industry and based on the sorting of the greek alphabet, the choice with .dev is arbitrary. That's what makes this a bikeshed. Either way of doing this is going to confuse and upset some of the consumers and there's no solid reason to say that the upset people are "wrong". At best you can say that they're late to the party.

...

So to be clear my proposal would be:

1.0dev1 < 1.0a1 < 1.0b1 < 1.0c1 < 1.0.pre1 < 1.0 < 1.0.post1

as opposed to the current:

1.0a1 < 1.0b1 < 1.0c1 < 1.0.dev1 < 1.0 < 1.0.post1

+1 for the proposal to change the PEP so that 'dev' versions are earlier than 'a' versions.

Note, just in case my chart is misleading, the proposal doesn't change all 'dev' versions to be earlier than the 'a' versions. '.dev' is a modifier. So it can be applied to any of the toplevel portions of the version. So to be more complete, the proposal is to do this:: 0.1 < 0.1.post1 < 1.0.dev1 # The .dev that's changing position < 1.0a1.dev1 < 1.0a1 < 1.0a1.post1 < 1.0b1.dev1 < 1.0b1 < 1.0b1.post1 < 1.0c1.dev1 < 1.0c1 < 1.0c1.post1 # The current PEP386 position of 1.0.dev1 < 1.0 < 1.0.post1 -Toshio

On Thu, 2012-09-27 at 16:15 -0700, Toshio Kuratomi wrote:

On Thu, Sep 27, 2012 at 05:00:56PM -0500, Brad Allen wrote:

...

On Thu, Sep 27, 2012 at 2:45 PM, Toshio Kuratomi wrote:

...

However, now that you mention it, ISTR that this is the way that it was originally and the lack of pre/dev in that position did bother another (set of?) the PEP authors enough that it was changed. For me, switching the search order of that particular piece is a bikeshed so I seem to have reallocated the memory for storing the arguments made in favor of shifting it to the current position :-)

Why is this perceived as a bikeshed issue? It seems like a discussion about substantial functionality which is tied to the semantics of these special tags. For many projects and organizations, terms like 'alpha' and 'beta' and 'dev' have meaning in a release context, and rearranging the order of these concepts has an impact.

dev has no universal defined relation to alpha, beta, etc in a release context.

The tools that people use to install a Python package sort it before alpha and have for many years. That seems to be a pretty good argument in favor, all other things considered. - C

On Thu, Sep 27, 2012 at 7:15 PM, Toshio Kuratomi wrote:

So to be more complete, the proposal is to do this::

0.1 < 0.1.post1 < 1.0.dev1 # The .dev that's changing position < 1.0a1.dev1 < 1.0a1 < 1.0a1.post1 < 1.0b1.dev1 < 1.0b1 < 1.0b1.post1 < 1.0c1.dev1 < 1.0c1 < 1.0c1.post1 # The current PEP386 position of 1.0.dev1 < 1.0 < 1.0.post1

This looks perfect to me, and very unambiguous. Allowing the .dev and .post modifiers at every release stage I think clears up the issue here better than any other proposal. +1. (This is also semantically the same as semver--just not syntactically the same. I like semver but I think we're stuck with the existing syntax which is fine.) Erik

On Thu, Sep 27, 2012 at 1:46 PM, Éric Araujo wrote:

Le 27/09/2012 11:59, Toshio Kuratomi a écrit :

...

* a, b, c, and rc are the typical alpha beta release candidate tags. We debated whether to allow the single letter versions or the long versions "alpha", "beta", "rc". In the end, which set of tags was used was a bikeshed argument so it was chosen by PEP-author decree. I personally felt that having both "c" and "rc" was counter to the goals of the PEP but we at least agreed that "rc" would intuitively sort after "c" so it wasn't as bad as having both "alpha" and "a" where there would be no intuitively correct sorting.

Well, c and rc should really compare equal in my opinion. _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

Does that mean p == np? Ruby gems use http://semver.org/

On Thu, Sep 27, 2012 at 5:58 PM, Fred Drake wrote:

On Thu, Sep 27, 2012 at 1:50 PM, Daniel Holth wrote:

...

Ruby gems use http://semver.org/

I don't remember if I was involved in the past discussions (it's possible), but I'm in favor of using (and requiring) semantic versioning, if only because we don't have to invent notation or interpretation.

+1 -- Benji York

On Thu, Sep 27, 2012 at 01:46:25PM -0400, Éric Araujo wrote:

Le 27/09/2012 11:59, Toshio Kuratomi a écrit :

...

* a, b, c, and rc are the typical alpha beta release candidate tags. We debated whether to allow the single letter versions or the long versions "alpha", "beta", "rc". In the end, which set of tags was used was a bikeshed argument so it was chosen by PEP-author decree. I personally felt that having both "c" and "rc" was counter to the goals of the PEP but we at least agreed that "rc" would intuitively sort after "c" so it wasn't as bad as having both "alpha" and "a" where there would be no intuitively correct sorting.

Well, c and rc should really compare equal in my opinion.

That would be a bad thing. What do you do in the face of a project releasing: foo-1.0c1 foo-1.0rc1 Does your tool get to download either one of those depending on who coded it, the timestamp for the upload, or input from the RNG? And saying that the foo project maintainers shouldn't do that because they're semantically the same is well ad good but in practice people do things that are wrong all the time. With a single version string inside of a package we can provide functions that can validate whether the version is correct or not as part of using the versions in the library to mitigate that. We cannot do the same thing with version strings from two separate releases of the package because where those releases are stored is site/project specific. Documenting that even though c and rc are menat to be semantically the same they should always sort "c" followed by "rc" protects you from these problems. -Toshio