[Distutils] setuptools 8 changes are great, but ...

[Chris Withers]

On 16/12/2014 12:02, Marius Gedminas wrote:
>> Is there something I’m not aware that is broken currently? I thought the
>> transition was going pretty smoothly overall considering that a core piece
>> of code inside of setuptools was touched.
>
> ~80 zope.* test builds are currently failing, for mysterious reasons due
> to setuptools 8.0.x not interacting well with zc.buildout:
> https://mail.zope.org/pipermail/zope-dev/2014-December/046509.html
>
> Here's a summary of the various errors:
> https://mail.zope.org/pipermail/zope-dev/2014-December/046508.html
>
> Newer builds also added a bunch of warnings of the form
> /var/lib/jenkins/jobs/zopetoolkit_trunk/workspace/lib/python2.7/site-packages/pkg_resources.py:2425:
> RuntimeWarning: 'zc.recipe.testrunner-1.0.5 ()' is being parsed as a
> legacy, non PEP 440, version. You may find odd behavior and sort order.
> In particular it will be sorted as less than 0.0. It is recommend to
> migrate to PEP 440 compatible versions.
>
> Recent threads on distutils-sig@ help explain some of what's happening.
> E.g. the 'zope.app.wsgi<3.11,<4.0dev,>=3.12' probably used to be
> interpreted as an "<3.11 or >=3.12" and just needs to be hunted down and
> replaced with "!=3.11.*", or something like that.
>
> (It's painful when you get requirement conflict errors with no
> indication about the source of those requirements.)

FWIW, I've also been seeing failures in all of my buildout-based library 
testing Jenkins jobs:

This one only on Windows and Python 2.7 only, NOT Python 2.6:

C:\Jenkins\workspace\checker-buildout\aeb5917b>C:\Python27\python.exe 
bootstrap.py
Downloading 
https://pypi.python.org/packages/source/s/setuptools/setuptools-8.0.4.zip
Extracting in c:\users\jenkins\appdata\local\temp\tmp1hy01w
Now working in 
c:\users\jenkins\appdata\local\temp\tmp1hy01w\setuptools-8.0.4
Building a Setuptools egg in c:\users\jenkins\appdata\local\temp\tmpzl12vj
warning: no files found matching 'entries*' under directory 
'setuptools\tests'
warning: no files found matching 'Makefile' under directory 'docs'
warning: no files found matching 'indexsidebar.html' under directory 'docs'
c:\users\jenkins\appdata\local\temp\tmpzl12vj\setuptools-8.0.4-py2.7.egg
Traceback (most recent call last):
   File "bootstrap.py", line 92, in <module>
     ez['use_setuptools'](**setup_args)
   File "<string>", line 140, in use_setuptools
   File "<string>", line 128, in _do_download
   File "build\bdist.win-amd64\egg\setuptools\__init__.py", line 5, in 
<module>
   File "C:\Python27\lib\distutils\core.py", line 20, in <module>
     from distutils.dist import Distribution
   File "C:\Python27\lib\distutils\dist.py", line 10, in <module>
     from email import message_from_file
ImportError: No module named email

...which I see on a couple of jobs, also this, on Python 3, Linux:

Downloading 
https://pypi.python.org/packages/source/s/setuptools/setuptools-8.0.1.zip
Extracting in /tmp/tmp71bne9
Now working in /tmp/tmp71bne9/setuptools-8.0.1
Building a Setuptools egg in /tmp/tmpdgs_c0
/tmp/tmpdgs_c0/setuptools-8.0.1-py3.3.egg
Creating directory 
'/var/lib/jenkins/slave/workspace/mush-buildout/0321cca2/bin'.
Creating directory 
'/var/lib/jenkins/slave/workspace/mush-buildout/0321cca2/parts'.
Creating directory 
'/var/lib/jenkins/slave/workspace/mush-buildout/0321cca2/develop-eggs'.
Generated script 
'/var/lib/jenkins/slave/workspace/mush-buildout/0321cca2/bin/buildout'.
/tmp/tmpdgs_c0/setuptools-8.0.1-py3.3.egg/pkg_resources.py:130: 
RuntimeWarning: You have iterated over the result of 
pkg_resources.parse_version. This is a legacy behavior which is 
inconsistent with the new version class introduced in setuptools 8.0. 
That class should be used directly instead of attempting to iterate over 
the result.
/var/lib/jenkins/.buildout/eggs/setuptools-8.0.1-py3.3.egg/pkg_resources.py:130: 
RuntimeWarning: You have iterated over the result of 
pkg_resources.parse_version. This is a legacy behavior which is 
inconsistent with the new version class introduced in setuptools 8.0. 
That class should be used directly instead of attempting to iterate over 
the result.
While:
   Installing.
   Checking for upgrades.
   Getting distribution for 'zc.buildout>=2.2.5'.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/buildout.py", 
line 1946, in main
     getattr(buildout, command)(args)
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/buildout.py", 
line 475, in install
     self._maybe_upgrade()
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/buildout.py", 
line 910, in _maybe_upgrade
     allow_hosts = self._allow_hosts
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/easy_install.py", 
line 844, in install
     return installer.install(specs, working_set)
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/easy_install.py", 
line 631, in install
     for_buildout_run=for_buildout_run):
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/easy_install.py", 
line 473, in _get_dist
     dist, avail = self._satisfied(requirement)
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/easy_install.py", 
line 268, in _satisfied
     best_available = self._obtain(req, source)
   File 
"/var/lib/jenkins/.buildout/eggs/zc.buildout-2.2.5-py3.3.egg/zc/buildout/easy_install.py", 
line 427, in _obtain
     if distv > bestv:
TypeError: unorderable types: SetuptoolsVersion() > tuple()

...and this one on Windows Python 3:

C:\Jenkins\workspace\mush-buildout\a1017537>C:\Python33\python.exe 
bootstrap.py
Downloading 
https://pypi.python.org/packages/source/s/setuptools/setuptools-8.0.1.zip
Extracting in c:\users\jenkins\appdata\local\temp\tmpkij7_u
Now working in 
c:\users\jenkins\appdata\local\temp\tmpkij7_u\setuptools-8.0.1
Building a Setuptools egg in c:\users\jenkins\appdata\local\temp\tmpgy_a7q
c:\users\jenkins\appdata\local\temp\tmpgy_a7q\setuptools-8.0.1-py3.3.egg
Traceback (most recent call last):
   File "<string>", line 138, in use_setuptools
ImportError: No module named 'pkg_resources'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "bootstrap.py", line 92, in <module>
     ez['use_setuptools'](**setup_args)
   File "<string>", line 140, in use_setuptools
   File "<string>", line 128, in _do_download
   File 
"c:\users\jenkins\appdata\local\temp\tmpgy_a7q\setuptools-8.0.1-py3.3.egg\setuptools\__init__.py", 
line 5, in <module>
   File "C:\Python33\lib\distutils\core.py", line 17, in <module>
     from distutils.dist import Distribution
   File "C:\Python33\lib\distutils\dist.py", line 15, in <module>
     from distutils.fancy_getopt import FancyGetopt, translate_longopt
   File "C:\Python33\lib\distutils\fancy_getopt.py", line 12, in <module>
     import getopt
ImportError: No module named 'getopt'

Links to these builds are here, if you're familiar with Jenkins:

http://jenkins.simplistix.co.uk/job/mush-buildout/
http://jenkins.simplistix.co.uk/job/checker-buildout/

cheers,

Chris