Hi Nick, Thanks for the detailed reply. I'll stick to PEP426 related topics in this mail. Generally speaking, PEP426 looks like good progress, but the biggest problem I see now is the lack of parallel installs for incompatible versions. On Mon, 2013-03-04 at 18:11 +1000, Nick Coghlan wrote:
On Mon, Mar 4, 2013 at 1:54 AM, Mark McLoughlin
wrote: I've tried to digest PEP426 and collected my thoughts below. Apologies if you read through them and they offer you nothing useful.
I'm trying to imagine the future we're driving towards here where OpenStack is no longer suffering and how PEP426 helped get us there
e.g. where
- Many libraries use semantic versioning and OpenStack specifies x.y compatible versions in their dependency lists. Incompatible changes only get made in x+N.y and OpenStack continues using the x.y+N.z versions.
Yep, PEP 426 pushes projects heavily in that direction by making it the path of least resistance. You *can* do something different if necessary, it's just less convenient than simply using the first 9 clauses of semantic versioning.
- All the libraries which don't use semantic versioning use another clear versioning scheme that allows OpenStack to avoid incompatible updates while still using compatible updates.
Yes, the PEP will explicitly encourage projects to document how to define dependencies if the default "compatible release" clause isn't appropriate.
Ok, understood. Is there a case for putting this information into the metadata? e.g. if this was version 1.2.3 of a library and the project's policy is to bump the major version if an incompatible API change is made, then you can do: Requires-Dist: foo (1.2) or: Requires-Dist: foo (1.2,>=1.2.3) or: Requires-Dist: foo (>=1.2.3,<2) but how about if foo-1.2.3 contained: Compatible-With: 1 and foo-2.0.0 contained: Compatible-With: 2 then this: Requires-Dist: foo (1.2) could reject 2.0.0 as satisfying the compatible release specification.
== PEP426 ==
=== Date Based Versioning ===
OpenStack uses date based versioning for its server components and we've begun using it for the Oslo libraries described above. PEP426 says:
Date based release numbers are explicitly excluded from compatibility with this scheme, as they hinder automatic translation to other versioning schemes, as well as preventing the adoption of semantic versioning without changing the name of the project. Accordingly, a leading release component greater than or equal to 1980 is an error.
This seems odd and the rationale seems week. It looks like an abritrary half-way house between requiring semantic versioning and allowing any scheme of increasing versions.
The assumption is that the version field will follow semantic versioning (that's not where the PEP started, it was originally non-committal on the matter like PEP 345 - however I've come to the conclusion that this assumption needs to be made explicit, but still need to update various parts of the PEP that are still wishy-washy about it). However, we can't actually enforce semantic versioning because we can't automatically detect if a project is adhering to those semantics. What we *can* enforce is the rejection of version numbers that obviously look like years rather than a semantic version.
I sympathise with wanting to do *something* here. Predictable instability was the first thing I listed above and semantic versioning would give that. However, I also described the approach I'm taking to predictable instability with the Oslo libraries. Any APIs we want to remove will be marked as deprecated for a year, then removed. We can encoded that as: oslo-config>=2013.1,<2014.1 if Compatible-With existed, we could do: Compatible-With: 2013.1,2013.2 If we used semantic versioning strictly, we'd bump the major release every 6 months (since every release may see APIs which were deprecated a year ago removed). Basically, PEP426 desires to enforce some meaning around the major number and API instability, while not enforcing a meaning of API stability for the micro version. I'm not sure it achieves a whole lot except making my life harder without actually forcing me into adopting a more predictable versioning scheme. And to be clear - this approach of removing APIs after a year is because we assume only other OpenStack projects are using Oslo libraries. If that changed, we'd probably keep deprecated APIs around for a lot longer.
=== Compatible Release Version Specifier ===
I know this is copied from Ruby's spermy operator, but it seems to be to be fairly useless (overly pessimistic, at least) with semantic versioning.
i.e. assuming x.y.z releases, where an increase in X means that incompatible changes have been made then:
~>1.2.3
unnecessarily limits you to the 1.2.y series of releases. Now, in an ideal world, you would just say:
~>1.2
and you're good for the entire 1.x.y but often you know what your app doesn't work with 1.2.2 and there's a fix in 1.2.3 that your app absolutely needs.
Handling that kind of situation is why version specifiers allow multiple clauses that are ANDed together:
mydependency (1.2, >= 1.2.3)
Thanks.
A random implementation detail I don't get here ... we use 'pip install -r' a lot in OpenStack and so have files with e.g.
PasteDeploy>=1.5.0
how will you specify compatible releases in these files? What's the equivalent of
PasteDeploy~>1.5.0
?
Note that the pkg_resources specifier format (no parens allowed) is NOT the same as the PEP 345/426 format (parens required). The former is designed to work with arbitrary version schemes, while the latter is explicitly constrained to version schemes that abide by the constraints described in PEP 386. I expect the setuptools/distribute developers will choose a tilde-based operator to represent compatible release clauses in the legacy formats (likely Ruby's ~>, although it could be something like ~= or ~~ instead).
Ok, figures. Cheers, Mark.