[Distutils] latest setuptools appears to require six in a breaking way

[Matthias Bussonnier]

> Just because it's on the release notes doesn't mean its not an issue.

Sorry if I miss expressed myself, let me retry:

If it is on the release notes, then it is likely because the
maintainers are aware. And if they released this version with this in
the release notes, then trying to fill a bug will likely result in it
being dismissed as duplicate.
I tried to be too concise.

I don't disagree that this change is likely to be quite painful, and I
have not contributed enough to setuptools to contradict the choice the
maintainers have had. I just linked to (and cited) the release notes,
gave a reason as I was aware (since yesterday, made /r/python top 10)
of why you were seeing what you are seeing.

I do understand the difficulty to upgrade all the CI script and
library. I've been in the process of upgrading our test suite and test
of all the packages I maintain to make sure we don't use
DeprecatedBehavior of 3.6 (Oh invalid backslash escape sequence is fun
in docstring that give example with windows Path). So yes that a
couple of weeks of low productivity. But at the same time that's the
price I chose to pay to support such a wide range of versions and have
sofware from 2 differents decades to work together.

I want to add that if your system is still using old pip/setuptools
version then it might soon break for other reasons. Setuptools < 24
(IIRC) and pip < 9 cannot handle Python 3 only Sdist correctly. So if
one of your libraries you use publish a Python 3 only sdist (which is
likely to happen at some point in time) then your CI will install the
latest python 3 only sdist on python 2 instead of the latest Python 2
compatible sdist. Which is likely to fail as well.

So at some point upgrading pip and setuptools as a first step of CI
integration might be anyway necessary.

Hope this problem will be either accepted or solved soon.

Thanks
-- 
M





On Wed, Jan 25, 2017 at 9:20 AM, Chris Withers <chris at simplistix.co.uk> wrote:
> On 25/01/2017 16:44, Matthias Bussonnier wrote:
>>
>> AFAICT this is not really an issue as this is on the release notes:
>>
>> https://setuptools.readthedocs.io/en/latest/history.html#v34-0-0
>
>
> Just because it's on the release notes doesn't mean its not an issue.
>
>>> #581: Instead of vendoring the growing list of dependencies that
>>> Setuptools requires to function, Setuptools now requires these dependencies
>>> just like any other project. Unlike other projects, however, Setuptools
>>> cannot rely on setup_requires to demand the dependencies it needs to install
>>> because its own machinery would be necessary to pull those dependencies if
>>> not present (a bootstrapping problem). As a result, Setuptools no longer
>>> supports self upgrade or installation in the general case. Instead, users
>>> are directed to use pip to install and upgrade using the wheel distributions
>>> of setuptools.
>
>
> For such a fundamental part of the python packaging ecosystem, that's a
> pretty disappointing change. Not supporting self-upgrade seems like a pretty
> massive regression, and the problems here are a result.
>
> None of the problems I've had involve explicit installation of setuptools,
> it seems to be dragged in as a dependency somewhere along the way. The lack
> of ability to self-update in that situation is going to cause major pain.
>
>>  >As discovered in #940, not all versions of pip will successfully
>> install Setuptools from its pre-built wheel. If you encounter issues
>> with “No module named six” or “No module named packaging”, especially
>> following a line “Running setup.py egg_info for package setuptools”,
>> then your pip is not new enough.
>
>
> Right, so what's the recommended one-step way to set up a virtualenv now in
> Py 2.6-3.6?
>
>>> There’s an additional issue in pip where setuptools is upgraded
>>> concurrently with other source packages, described in pip #4253. The
>>> proposed workaround is to always upgrade Setuptools first prior to upgrading
>>> other packages that would upgrade Setuptools.
>
>
> That workaround is quite painful if you maintain any number of libraries
> that have CI scripts that will need to be hacked up to support the
> workaround and then reverted after the issue is finally fixed.
>
> Sorry if this comes across as negative, I know it's not always easy to judge
> the outcome of changes before a big release, but I do hope something can be
> done to mitigate the pain of library maintainers doing the right thing and
> running CI...
>
> cheers,
>
> Chris