[Distutils] Local version identifiers from PEP 440 in practice

Donald Stufft donald at stufft.io
Wed Dec 17 02:49:24 CET 2014 

> On Dec 16, 2014, at 8:33 PM, Maurits van Rees <m.van.rees at zestsoftware.nl> wrote:
> 
> Donald Stufft schreef op 17-12-14 01:54:
>> 
>>> On Dec 16, 2014, at 7:46 PM, Maurits van Rees <m.van.rees at zestsoftware.nl> wrote:
>>> 
>>> Maurits van Rees schreef op 17-12-14 00:53:
>>>> I have created a very basic python project called 'myproject'.  It
>>>> does nothing.  I have released a few versions here:
>>>> http://pypi.zestsoftware.nl/public/packagingtest/
>>> 
>>> I have now also distributed myproject version 1.1.  (This has a
>>> base.jinja2 file and requires Jinja2[i18n], which I need for checking
>>> a corner case; I may report that later).
>>> 
>>> Installing 1.1 unexpectedly gives problems, both with pip and
>>> zc.buildout.
>>> 
>>> - zc.buildout 2.2.5 and setuptools 7.0: all is well.
>>> 
>>> - zc.buildout 2.3.1 and setuptools 8.0.4:
>>> 
>>>  * When I specify 'myproject = 1.1' in the buildout config, myproject
>>>    does not get updated.  It sticks at version 1.1+maurits.3.
>> 
>> This isn’t a bug, assuming that buildout is translating ``my project = 1.1``
>> to something like ==1.1.
> 
> That is indeed what buildout does.  Note that buildout config files are basically parsed by ConfigParser, so '==' is not allowed there.  That is why buildout uses a different notation here.
> 
>> Local versions are used to indicate something that
>> is compatible with 1.1 and they sort as newer than the same version without
>> the local version. The reason they match for a ``==1.1`` is because you want
>> downstream distributors like Debian to be able to set their versions to 1.0+debian.1
>> without breaking things for people who do ``==1.1``.
> 
> That is most definitely not what I want.  If I tell setuptools or pip or buildout that I want version X, then I don't want roughly version X, but I want exactly version X.
> 
> When I request 1.0rc1 and I get 1.0, that is wrong.
> When I request 1.0 and I get 1.0rc1, that is wrong.
> 
> To me, the same is true for local version identifiers.
> 
> To me, this is the basis for repeatable deployments, knowing that given a buildout file or a pip requirements file you get the same versions and the same code that you got when you tried it a month earlier.  It avoids "it works on my machine" when I get an error that a colleague does not get.
> 
> 
> For clarity, how I use internal releases, so how I would want to use local version identifiers, is:
> 
> 1. Use largetrout 1.0 in a project.
> 
> 2. Notice a bug.
> 
> 3. See that the bug is already fixed in development, but there is no release yet.
> 
> 4. Checkout largetrout dev, setup.py version is 1.1.dev0.
> 
> 5. Set version to 1.1+internal1 (an internal release of a snapshot of the code that is expected to end up in 1.1).
> 
> 6. Make an internal release and use it.
> 
> 7. Five more bugs get fixed in largetrout.
> 
> 8. largetrout 1.1 is released.
> 
> Now when I tell pip or buildout to use largetrout 1.1 I definitely want those five bug fixes from step 7 included.

So the *primary* use case that motivated local versions is things like when Debian patches a copy of a project they can indicate that they’ve done so by changing the version to 1.0+dfsg1 or so instead of 1.0. A related use case is the one you’re talking about. However the primary use case is what influenced the fact that ==1.1 matches ==1.0+foobar.

Important to note, is that ==1.0+foobar will only install that patched version, not any 1.0 version. You can also approximate the kind of pinning you want with the === (which is really the arbitrary equality indicator, which is generally used for people who want to install a version like “dog” which we can’t parse). It’s possible that we could add some sort of a “None” indicator for local versions that says “1.0 and exactly 1.0” though I’m not sure how off the top of my head (Maybe ==1.0+).

Another thing though, what you probably want to do is something like 1.1.dev0+internal1, which will sort as older than 1.1 whenever it is actually released.

> 
>>> - pip 1.5.6, setuptools 7.0:
>>> 
>>>  * pip warns about three different versions:
>>>    $ pip install -U -f http://pypi.zestsoftware.nl/public/packagingtest/ myproject==1.1
>>>    Downloading/unpacking myproject==1.1
>>>      Downloading myproject-1.1+maurits.3.zip
>>>      Running setup.py (path:/Users/mauritsvanrees/tmp/venv-older/build/myproject/setup.py) egg_info for package myproject
>>>      Requested myproject==1.1, but installing version 1.1-maurits.3
>>>  * The warning is correct: not 1.1, but 1.1-maurits.3 is installed.
>> 
>> That warning might actually be from a stale build directory laying around and not related to the local version at all.
> 
> Let me check.  Nope, the same happens with a fresh virtualenv:
> 
> mauritsvanrees at procyon:tmp $ virtualenv-2.7 venv-older
> Using real prefix '/Users/mauritsvanrees/buildout/python2.7/parts/opt'
> New python executable in venv-older/bin/python2.7
> Also creating executable in venv-older/bin/python
> Installing setuptools, pip...done.
> mauritsvanrees at procyon:tmp $ cd venv-older
> mauritsvanrees at procyon:venv-older $ . bin/activate
> (venv-older)mauritsvanrees at procyon:venv-older $ pip install -U setuptools==7.0  # this is optional
> Downloading/unpacking setuptools==7.0
>  Downloading setuptools-7.0-py2.py3-none-any.whl (534kB): 534kB downloaded
> Installing collected packages: setuptools
>  Found existing installation: setuptools 3.6
>    Uninstalling setuptools:
>      Successfully uninstalled setuptools
> Successfully installed setuptools
> Cleaning up...
> (venv-older)mauritsvanrees at procyon:venv-older $ pip list
> pip (1.5.6)
> setuptools (7.0)
> wsgiref (0.1.2)
> (venv-older)mauritsvanrees at procyon:venv-older $ pip install -U -f http://pypi.zestsoftware.nl/public/packagingtest/ myproject==1.1
> Downloading/unpacking myproject==1.1
>  http://pypi.zestsoftware.nl/public/packagingtest/ uses an insecure transport scheme (http). Consider using https if pypi.zestsoftware.nl has it available
>  Downloading myproject-1.1+maurits.3.zip
>  Running setup.py (path:/Users/mauritsvanrees/tmp/venv-older/build/myproject/setup.py) egg_info for package myproject
> 
>    warning: no files found matching '*rst'
>    warning: no previously-included files matching '*.pyc' found anywhere in distribution
>  Requested myproject==1.1, but installing version 1.1-maurits.3
> Installing collected packages: myproject
>  Running setup.py install for myproject
> 
>    warning: no files found matching '*rst'
>    warning: no previously-included files matching '*.pyc' found anywhere in distribution
> Successfully installed myproject
> Cleaning up...
> (venv-older)mauritsvanrees at procyon:venv-older $ ls -1 lib/python2.7/site-packages/
> _markerlib
> easy_install.py
> easy_install.pyc
> myproject
> myproject-1.1_maurits.3-py2.7.egg-info
> pip
> pip-1.5.6.dist-info
> pkg_resources.py
> pkg_resources.pyc
> setuptools
> setuptools-7.0.dist-info
> 
> 
> -- 
> Maurits van Rees: http://maurits.vanrees.org/
> Zest Software: http://zestsoftware.nl
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

More information about the Distutils-SIG mailing list