On 02/23/2013 08:08 PM, Nick Coghlan wrote:

On Sun, Feb 24, 2013 at 4:43 AM, Daniel Holth wrote:

...

There simply must be a way to spell "equals" that means "equals". It will be used when that so-called security release broke my application that integrates said library in a way that doesn't even expose the flaw.

Plone depends on hundreds of libraries that probably only use >= or no version at all in their setup.py. Then the buildout.cfg ships with the == versions of all the libraries that the Plone release team actually tested. == is not the common plague that you fear.

The core problem with making "==" strict is that it either makes "!=" useless by allowing "(!=1.3)" to match "1.3.1", or it breaks the invariant that "!=" is the logical inverse of "==", by having both "(==1.3)" and "(!=1.3)" reject "1.3.1". I don't consider either acceptable, which is why I switched the PEP to simple string prefix matching for both [snip] If a true exact version match is needed in order to be completely certain about avoiding potentially broken upstream releases, then you can use "(X.Y, < X.Y.post0)" rather than "(==X.Y)". Really though, that level of pre-emptive quality assurance is better handled by using a private curated index (or even a per-application index) where you don't allow new versions to be uploaded until you've already tested them.

If PEP 426 dependency version requirements are meant to support applications as well as libraries, I very much agree with Daniel that a real, strict == is critical, and should be the default behavior of ==. I don't think "it makes != useless" is a sufficient argument to make == behave surprisingly; in my experience, the need for != is quite rare, and it makes more sense to require multi-clause workarounds for !=, rather than for the simple "this is what I want, don't you dare pick anything else" case, which is quite common. If you are dead-set against making == strict by default, then I think we need a (gulp) === or similar. Requiring multi-clause workarounds or curated package archives for strict dependency pinning is unacceptable, IMO. Carl

I realised globbing won't work easily, due to alpha/beta/release candidates missing the preceding dot, so 1.1.1.* wouldn't match versions like 1.1.1c1, while leaving the dot out would mean 1.1.1* matching versions like 1.1.11 (it also means the old PEP 345 default behaviour and the current "==" behaviour aren't *quite* simple prefix matching, since they shouldn't match a trailing numeric component with extra digits). Accordingly, I'm considering adding "is" as the comparison operator to say "this version, exactly this version, accept no substitutes". Cheers, Nick. On 26 Feb 2013 08:42, "Daniel Holth" wrote:

We have a potential solution from node which is to allow limited globs in version matches 1.0.* On Feb 25, 2013 3:04 PM, "Carl Meyer" wrote:

...

On 02/23/2013 08:08 PM, Nick Coghlan wrote:

...

On Sun, Feb 24, 2013 at 4:43 AM, Daniel Holth wrote:

...

There simply must be a way to spell "equals" that means "equals". It will be used when that so-called security release broke my application that integrates said library in a way that doesn't even expose the flaw.

Plone depends on hundreds of libraries that probably only use >= or no version at all in their setup.py. Then the buildout.cfg ships with the == versions of all the libraries that the Plone release team actually tested. == is not the common plague that you fear.

The core problem with making "==" strict is that it either makes "!=" useless by allowing "(!=1.3)" to match "1.3.1", or it breaks the invariant that "!=" is the logical inverse of "==", by having both "(==1.3)" and "(!=1.3)" reject "1.3.1". I don't consider either acceptable, which is why I switched the PEP to simple string prefix matching for both [snip] If a true exact version match is needed in order to be completely certain about avoiding potentially broken upstream releases, then you can use "(X.Y, < X.Y.post0)" rather than "(==X.Y)". Really though, that level of pre-emptive quality assurance is better handled by using a private curated index (or even a per-application index) where you don't allow new versions to be uploaded until you've already tested them.

If PEP 426 dependency version requirements are meant to support applications as well as libraries, I very much agree with Daniel that a real, strict == is critical, and should be the default behavior of ==. I don't think "it makes != useless" is a sufficient argument to make == behave surprisingly; in my experience, the need for != is quite rare, and it makes more sense to require multi-clause workarounds for !=, rather than for the simple "this is what I want, don't you dare pick anything else" case, which is quite common.

If you are dead-set against making == strict by default, then I think we need a (gulp) === or similar. Requiring multi-clause workarounds or curated package archives for strict dependency pinning is unacceptable, IMO.

Carl _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

_______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

On Tue, Feb 26, 2013 at 11:06 AM, Donald Stufft wrote:

!=1.3 allowing 1.3.1 makes sense to me. 1.3 is equivalent to 1.3.0, 1.3.1 != 1.3.0.

Nope, the PEP explicitly disclaims the historical equivalence between 1.3 and 1.3.0. It has to, because they're very different when it comes to what the PEP now calls a "compatible release clause" (http://www.python.org/dev/peps/pep-0426/#compatible-release), Ruby gems call a "pessimistic version constraint" (http://docs.rubygems.org/read/chapter/16#page74), PHP composer calls "next significant release" (http://getcomposer.org/doc/01-basic-usage.md#package-versions) and Node.js npm calls "Tilde version range" (https://npmjs.org/doc/json.html) (Note: I also checked CPAN version ordering to see if it had an equivalent operation, but Perl's approach appears to be closer to the way pkg_resources handles ordering: http://search.cpan.org/~jpeacock/version-0.9901/lib/version.pod#How_to_compa...) However, while numeric maintenance releases are part of the problem here, the real issue is correctly handling *post* releases. Does "== 1.3" allow "1.3.post1"? Does "!= 1.3" allow "1.3.post1"? Both? Neither? If you use strict string matching for == and !=, then "!= 1.3" will allow "1.3.post1", which is utterly insane given the PEP's recommended usage of post releases solely for non-functional changes like tweaking the release notes or project metadata. Leaving == as strict and making != prefix based breaks the fundamental relationship between equality and inequality checks, so that's also not a reasonable option. That leaves making both == and != prefix based as the most reasonable alternative, as "!= 1.3" will then correctly reject "1.3.post1" and the appropriate logical relationship is maintained between the two operators. However, you then need a way to spell an *exact* version request (for use with tools like zc.buildout and pip freeze that are designed to snapshot an application configuration rather than for dependency publication in an index). My plan for that use case is to allow "is" as a comparison operator that means exactly what it says: the version required is precisely the specified version, with no prefix matching or release compatibility assessments allowed. That gives a natural progression in dependency constraints from more permissive to less permissive: Compatible version: some-dist (X.Y) # roughly equivalent to (>= X.Y, < X+1.dev0) Version equality: some-dist (== X.Y) # roughly equivalent to (>= X.Y, < X.Y+1.dev0) Version identity: some-dist (is X.Y) # must be exactly X.Y All three clauses also have clearly defined use cases: 1. Use compatible version clauses in published metadata for stable dependencies with a good backwards compatibility policy 2. Use version equality clauses in published metadata for dependencies without a good backwards compatibility policy 3. Use version identity clauses for application and deployment dependency snapshots Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Tuesday, February 26, 2013 at 7:08 AM, Michael Foord wrote:

On 26 February 2013 03:22, Nick Coghlan wrote:

...

On Tue, Feb 26, 2013 at 11:06 AM, Donald Stufft wrote:

...

!=1.3 allowing 1.3.1 makes sense to me. 1.3 is equivalent to 1.3.0, 1.3.1 != 1.3.0.

Nope, the PEP explicitly disclaims the historical equivalence between 1.3 and 1.3.0. It has to, because they're very different when it comes to what the PEP now calls a "compatible release clause" (http://www.python.org/dev/peps/pep-0426/#compatible-release), Ruby gems call a "pessimistic version constraint" (http://docs.rubygems.org/read/chapter/16#page74), PHP composer calls "next significant release" (http://getcomposer.org/doc/01-basic-usage.md#package-versions) and Node.js npm calls "Tilde version range" (https://npmjs.org/doc/json.html) (Note: I also checked CPAN version ordering to see if it had an equivalent operation, but Perl's approach appears to be closer to the way pkg_resources handles ordering: http://search.cpan.org/~jpeacock/version-0.9901/lib/version.pod#How_to_compa...)

However, while numeric maintenance releases are part of the problem here, the real issue is correctly handling *post* releases.

Does "== 1.3" allow "1.3.post1"? Does "!= 1.3" allow "1.3.post1"? Both? Neither?

If you use strict string matching for == and !=, then "!= 1.3" will allow "1.3.post1", which is utterly insane given the PEP's recommended usage of post releases solely for non-functional changes like tweaking the release notes or project metadata.

Leaving == as strict and making != prefix based breaks the fundamental relationship between equality and inequality checks, so that's also not a reasonable option.

That leaves making both == and != prefix based as the most reasonable alternative, as "!= 1.3" will then correctly reject "1.3.post1" and the appropriate logical relationship is maintained between the two operators.

However, you then need a way to spell an *exact* version request (for use with tools like zc.buildout and pip freeze that are designed to snapshot an application configuration rather than for dependency publication in an index).

The current tools are strict with regards to equality - and in dependency pinning I would see it as *surprising* that specifying "==1.3" installs some version that isn't precisely 1.3. Having "==" mean approximately-equal, instead of the-same-as, seems like a mistake to me.

Michael

I agree with this, == should mean EXACTLY 1.3, != should mean EXACTLY NOT 1.3. However because these aren't strings and are versions we can understand that: 1.3 == 1.3.0 == 1.3.0.0 == 1.3.0.0.0 etc etc.

...

My plan for that use case is to allow "is" as a comparison operator that means exactly what it says: the version required is precisely the specified version, with no prefix matching or release compatibility assessments allowed.

That gives a natural progression in dependency constraints from more permissive to less permissive:

Compatible version: some-dist (X.Y) # roughly equivalent to (>= X.Y, < X+1.dev0) Version equality: some-dist (== X.Y) # roughly equivalent to (>= X.Y, < X.Y+1.dev0) Version identity: some-dist (is X.Y) # must be exactly X.Y

All three clauses also have clearly defined use cases:

1. Use compatible version clauses in published metadata for stable dependencies with a good backwards compatibility policy 2. Use version equality clauses in published metadata for dependencies without a good backwards compatibility policy 3. Use version identity clauses for application and deployment dependency snapshots

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com (mailto:ncoghlan@gmail.com) | Brisbane, Australia _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org (mailto:Distutils-SIG@python.org) http://mail.python.org/mailman/listinfo/distutils-sig

-- http://www.voidspace.org.uk/

May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On Tuesday, February 26, 2013 at 8:17 AM, Nick Coghlan wrote:

I *don't* like the idea of (== 1.3) and (== 1.3.0) being equivalent when (1.3) and (1.3.0) are substantially different, though. Instead, I'll reinstate a variant of the commentary from PEP 386 that pointed out the value of always publishing releases with a consistent number of components by including the trailing ".0".

I think it makes absolute sense. These aren't strings they are version numbers. 1.0 == 1.00 == 1.00 == 1.000, adding more zeros doesn't change the semantics. It also matches the existing practices of setuptools.

On Tue, Feb 26, 2013 at 11:23 PM, Donald Stufft wrote:

On Tuesday, February 26, 2013 at 8:17 AM, Nick Coghlan wrote:

I *don't* like the idea of (== 1.3) and (== 1.3.0) being equivalent when (1.3) and (1.3.0) are substantially different, though. Instead, I'll reinstate a variant of the commentary from PEP 386 that pointed out the value of always publishing releases with a consistent number of components by including the trailing ".0".

I think it makes absolute sense. These aren't strings they are version numbers.

1.0 == 1.00 == 1.00 == 1.000, adding more zeros doesn't change the semantics.

The number of components matters, precisely in the "significant digits" sense that Chris mentions. In a compatible release clause, 1.3 is a declaration of compatibility with everything less than 2.0, whereas 1.3.0 limits the compatibility to versions less than 1.4.0 If 1.3 and 1.3.0 mean different things in that context (as they should), I'm *not* going to approve a standard that makes them the same when you stick an "==" sign in front of them. Remember, part of the goals of both PEP 386 and PEP 426 is to tighten up cases where setuptools is considered too permissive, because it's guessing in the face of ambiguity. "Trailing zeroes should be considered implied" when comparing versions is one such guess. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Tue, Feb 26, 2013 at 11:01 AM, Donald Stufft wrote:

On Tuesday, February 26, 2013 at 10:10 AM, Nick Coghlan wrote:

Remember, part of the goals of both PEP 386 and PEP 426 is to tighten up cases where setuptools is considered too permissive, because it's guessing in the face of ambiguity. "Trailing zeroes should be considered implied" when comparing versions is one such guess.

It goes against the common usage in the Python community.

* 135869 total releases * 66528 total releases that have at least 3 version levels (A.B.C) * 20244 total releases that have at least 3 version levels where A.B and A.B.0 appear to be equivalent[1]. * 86 total releases using at least 3 version levels where A.B and A.B.0 appear to be different[2].

Would break the assumptions made in ~30% of the releases using at least 3 version levels, or 15% of all releases. Furthermore A.B == A.B.0 being equivalent is a matter of opinion with no clear answer. I would be inclined to say that not breaking it for the 15% of all releases is a greater net benefit here then anyones notion of "right" unless there is a clear benefit.

For what it's worth, the implementation of PEP386 treats them as equal as well, choosing to pad the shorter one out to whatever the requisite number of 0's is required to make the number of components equal.

On a tangential note PyPI should probably not be allowing you to upload both a A.B and an A.B.0, or at the very least if we want PyPI to continue allowing it then we need to be explicit about how that should be handled in either scheme.

With "prefix equals" would the following evaluate to true? 1.4 == 1 == 1.3.1

On Wed, Feb 27, 2013 at 2:07 AM, Daniel Holth wrote:

With "prefix equals" would the following evaluate to true?

1.4 == 1 == 1.3.1

I guess that's the other advantage of the wildcard notation - it makes it very clear something special is going on :) For the ".0" expansion, I like Donald's explanation that when two numbers with an uneven number of components are compared with == or !=, the shorter one is expanded with ".0" elements until they have the same number. That gives the following comparison clauses: Compatible version: some-dist (X.Y) # Expects (>= X.Y, < X+1.dev0) Wildcard version: some-dist (== X.Y.*) # Expects (>= X.Y, < X.Y+1.dev0) Exact version: some-dist (== X.Y) # Expects X.Y, allows extra ".0" suffixes I can live with that. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Tue, Feb 26, 2013 at 2:53 PM, Chris Jerdonek wrote:

On Tue, Feb 26, 2013 at 8:01 AM, Donald Stufft wrote:

...

On Tuesday, February 26, 2013 at 10:10 AM, Nick Coghlan wrote:

Remember, part of the goals of both PEP 386 and PEP 426 is to tighten up cases where setuptools is considered too permissive, because it's guessing in the face of ambiguity. "Trailing zeroes should be considered implied" when comparing versions is one such guess.

It goes against the common usage in the Python community.

* 135869 total releases * 66528 total releases that have at least 3 version levels (A.B.C) * 20244 total releases that have at least 3 version levels where A.B and A.B.0 appear to be equivalent[1]. * 86 total releases using at least 3 version levels where A.B and A.B.0 appear to be different[2].

Would break the assumptions made in ~30% of the releases using at least 3 version levels, or 15% of all releases. Furthermore A.B == A.B.0 being equivalent is a matter of opinion with no clear answer. I would be inclined to say that not breaking it for the 15% of all releases is a greater net benefit here then anyones notion of "right" unless there is a clear benefit.

For what it's worth, the implementation of PEP386 treats them as equal as well, choosing to pad the shorter one out to whatever the requisite number of 0's is required to make the number of components equal.

On a tangential note PyPI should probably not be allowing you to upload both a A.B and an A.B.0, or at the very least if we want PyPI to continue allowing it then we need to be explicit about how that should be handled in either scheme.

There is a similar issue with leading zeroes. For example, I was able to upload to PyPI versions both of the form A.B.2 and A.B.02. The PEP should probably be explicit and state that any N must be the unique integer representation, and state how that case should be handled if violated (with respect to equality, etc).

Is it easy to check if any releases on PyPI have versions with leading zeroes?

--Chris

We could use an official PyPI normalization for both dist names and versions. PyPI reportedly at least prohibits dists that differ only in case. PEP 427 converts anything that is not a letter or a digit or . into _, extending the setuptools scheme by allowing Unicode letters and digits. Unfortunately this is unkind to my favorite dist https://crate.io/packages/%E2%98%83/ which Donald still hasn't written any code for.

On Tuesday, February 26, 2013 at 2:53 PM, Chris Jerdonek wrote:

On Tue, Feb 26, 2013 at 8:01 AM, Donald Stufft wrote:

...

On Tuesday, February 26, 2013 at 10:10 AM, Nick Coghlan wrote:

Remember, part of the goals of both PEP 386 and PEP 426 is to tighten up cases where setuptools is considered too permissive, because it's guessing in the face of ambiguity. "Trailing zeroes should be considered implied" when comparing versions is one such guess.

It goes against the common usage in the Python community.

* 135869 total releases * 66528 total releases that have at least 3 version levels (A.B.C) * 20244 total releases that have at least 3 version levels where A.B and A.B.0 appear to be equivalent[1]. * 86 total releases using at least 3 version levels where A.B and A.B.0 appear to be different[2].

Would break the assumptions made in ~30% of the releases using at least 3 version levels, or 15% of all releases. Furthermore A.B == A.B.0 being equivalent is a matter of opinion with no clear answer. I would be inclined to say that not breaking it for the 15% of all releases is a greater net benefit here then anyones notion of "right" unless there is a clear benefit.

For what it's worth, the implementation of PEP386 treats them as equal as well, choosing to pad the shorter one out to whatever the requisite number of 0's is required to make the number of components equal.

On a tangential note PyPI should probably not be allowing you to upload both a A.B and an A.B.0, or at the very least if we want PyPI to continue allowing it then we need to be explicit about how that should be handled in either scheme.

There is a similar issue with leading zeroes. For example, I was able to upload to PyPI versions both of the form A.B.2 and A.B.02. The PEP should probably be explicit and state that any N must be the unique integer representation, and state how that case should be handled if violated (with respect to equality, etc).

Is it easy to check if any releases on PyPI have versions with leading zeroes? In [45]: l = [x for x in releases if any([y for y in x[1].split(".") if y.startswith("0") and y.isdigit() and not y.endswith("0")])]

In [46]: len(l) Out[46]: 1162 Note this doesn't check if they have confusing versions, (e.g. 1.1 and 1.01) just if they have a segment with a leading 0.

--Chris

On Tuesday, February 26, 2013 at 7:06 PM, Chris Jerdonek wrote:

On Tue, Feb 26, 2013 at 2:08 PM, Donald Stufft wrote:

...

In [45]: l = [x for x in releases if any([y for y in x[1].split(".") if y.startswith("0") and y.isdigit() and not y.endswith("0")])]

In [46]: len(l) Out[46]: 1162

Note this doesn't check if they have confusing versions, (e.g. 1.1 and 1.01) just if they have a segment with a leading 0.

Thanks. Probably not worth doing again, but should the last part be y != "0" to include cases like "010" and "00"?

--Chris I happened to still have that console open,

In [49]: len([x for x in releases if any([y for y in x[1].split(".") if y.startswith("0") and y.isdigit() and y != "0"])]) Out[49]: 1236 In [52]: len([x for x in releases if any([y for y in x[1].split(".") if y.startswith("0") and y.isdigit() and int(y) != 0])]) Out[52]: 1177