[Distutils] Local version identifiers from PEP 440 in practice

Wed Dec 17 02:33:55 CET 2014

Donald Stufft schreef op 17-12-14 01:54:
>
>> On Dec 16, 2014, at 7:46 PM, Maurits van Rees <m.van.rees at zestsoftware.nl> wrote:
>>
>> Maurits van Rees schreef op 17-12-14 00:53:
>>> I have created a very basic python project called 'myproject'.  It
>>> does nothing.  I have released a few versions here:
>>> http://pypi.zestsoftware.nl/public/packagingtest/
>>
>> I have now also distributed myproject version 1.1.  (This has a
>> base.jinja2 file and requires Jinja2[i18n], which I need for checking
>> a corner case; I may report that later).
>>
>> Installing 1.1 unexpectedly gives problems, both with pip and
>> zc.buildout.
>>
>> - zc.buildout 2.2.5 and setuptools 7.0: all is well.
>>
>> - zc.buildout 2.3.1 and setuptools 8.0.4:
>>
>>   * When I specify 'myproject = 1.1' in the buildout config, myproject
>>     does not get updated.  It sticks at version 1.1+maurits.3.
>
> This isn’t a bug, assuming that buildout is translating ``my project = 1.1``
> to something like ==1.1.

That is indeed what buildout does.  Note that buildout config files are 
basically parsed by ConfigParser, so '==' is not allowed there.  That is 
why buildout uses a different notation here.

> Local versions are used to indicate something that
> is compatible with 1.1 and they sort as newer than the same version without
> the local version. The reason they match for a ``==1.1`` is because you want
> downstream distributors like Debian to be able to set their versions to 1.0+debian.1
> without breaking things for people who do ``==1.1``.

That is most definitely not what I want.  If I tell setuptools or pip or 
buildout that I want version X, then I don't want roughly version X, but 
I want exactly version X.

When I request 1.0rc1 and I get 1.0, that is wrong.
When I request 1.0 and I get 1.0rc1, that is wrong.

To me, the same is true for local version identifiers.

To me, this is the basis for repeatable deployments, knowing that given 
a buildout file or a pip requirements file you get the same versions and 
the same code that you got when you tried it a month earlier.  It avoids 
"it works on my machine" when I get an error that a colleague does not get.

For clarity, how I use internal releases, so how I would want to use 
local version identifiers, is:

1. Use largetrout 1.0 in a project.

2. Notice a bug.

3. See that the bug is already fixed in development, but there is no 
release yet.

4. Checkout largetrout dev, setup.py version is 1.1.dev0.

5. Set version to 1.1+internal1 (an internal release of a snapshot of 
the code that is expected to end up in 1.1).

6. Make an internal release and use it.

7. Five more bugs get fixed in largetrout.

8. largetrout 1.1 is released.

Now when I tell pip or buildout to use largetrout 1.1 I definitely want 
those five bug fixes from step 7 included.

>> - pip 1.5.6, setuptools 7.0:
>>
>>   * pip warns about three different versions:
>>     $ pip install -U -f http://pypi.zestsoftware.nl/public/packagingtest/ myproject==1.1
>>     Downloading/unpacking myproject==1.1
>>       Downloading myproject-1.1+maurits.3.zip
>>       Running setup.py (path:/Users/mauritsvanrees/tmp/venv-older/build/myproject/setup.py) egg_info for package myproject
>>       Requested myproject==1.1, but installing version 1.1-maurits.3
>>   * The warning is correct: not 1.1, but 1.1-maurits.3 is installed.
>
> That warning might actually be from a stale build directory laying around and not related to the local version at all.

Let me check.  Nope, the same happens with a fresh virtualenv:

mauritsvanrees at procyon:tmp $ virtualenv-2.7 venv-older
Using real prefix '/Users/mauritsvanrees/buildout/python2.7/parts/opt'
New python executable in venv-older/bin/python2.7
Also creating executable in venv-older/bin/python
Installing setuptools, pip...done.
mauritsvanrees at procyon:tmp $ cd venv-older
mauritsvanrees at procyon:venv-older $ . bin/activate
(venv-older)mauritsvanrees at procyon:venv-older $ pip install -U 
setuptools==7.0  # this is optional
Downloading/unpacking setuptools==7.0
   Downloading setuptools-7.0-py2.py3-none-any.whl (534kB): 534kB downloaded
Installing collected packages: setuptools
   Found existing installation: setuptools 3.6
     Uninstalling setuptools:
       Successfully uninstalled setuptools
Successfully installed setuptools
Cleaning up...
(venv-older)mauritsvanrees at procyon:venv-older $ pip list
pip (1.5.6)
setuptools (7.0)
wsgiref (0.1.2)
(venv-older)mauritsvanrees at procyon:venv-older $ pip install -U -f 
http://pypi.zestsoftware.nl/public/packagingtest/ myproject==1.1
Downloading/unpacking myproject==1.1
   http://pypi.zestsoftware.nl/public/packagingtest/ uses an insecure 
transport scheme (http). Consider using https if pypi.zestsoftware.nl 
has it available
   Downloading myproject-1.1+maurits.3.zip
   Running setup.py 
(path:/Users/mauritsvanrees/tmp/venv-older/build/myproject/setup.py) 
egg_info for package myproject

     warning: no files found matching '*rst'
     warning: no previously-included files matching '*.pyc' found 
anywhere in distribution
   Requested myproject==1.1, but installing version 1.1-maurits.3
Installing collected packages: myproject
   Running setup.py install for myproject

     warning: no files found matching '*rst'
     warning: no previously-included files matching '*.pyc' found 
anywhere in distribution
Successfully installed myproject
Cleaning up...
(venv-older)mauritsvanrees at procyon:venv-older $ ls -1 
lib/python2.7/site-packages/
_markerlib
easy_install.py
easy_install.pyc
myproject
myproject-1.1_maurits.3-py2.7.egg-info
pip
pip-1.5.6.dist-info
pkg_resources.py
pkg_resources.pyc
setuptools
setuptools-7.0.dist-info

-- 
Maurits van Rees: http://maurits.vanrees.org/
Zest Software: http://zestsoftware.nl