[Distutils] can someone explain why I have started to get these pip failures

[Jeremy Stanley]

On 2016-07-14 16:01:23 +0100 (+0100), Robin Becker wrote:
> On 14/07/2016 15:41, Ian Cordasco wrote:
> >Try:
> >
> >pip install --force-reinstall setuptools -U
> >
> 
> I didn't do the force-reinstall and for some reason when I cleaned both
> ~/.cache/pip and ~/.pip the pip install -r requirements.txt did work.
> 
> I have tried various solutions proposed in the past eg
> 
> sudo apt-get install --reinstall python-pkg-resources
> 
> but nothing seems to work.
> 
> I did the cache cleanups in desperation mode.
> 
> I did try pip install -U setuptools, but it says it is up to date.
> 
> If this might be a case of the underlying python changing on the server I
> have turned off automatic security updates.
> 
> I would like to try and understand this happens as then I might have some
> wya of fixing it.

You really should avoid mixing pip-installed packages in the system
context with distro-provided Python libraries, otherwise you will
run into these sorts of issues constantly. I help maintain some
very, very large test infrastructure for Python-based tools and
libraries: in scenarios where we use pip to install anything
system-wide we first make sure to scrub every last distro-provided
Python library from the system along with any other Python-based
applications that might depend on them, and only then we bootstrap
pip completely independent of distro packaging (downloading and
running get-pip.py). Also whenever possible, we instead rely on pip
install within virtualenvs _without_ --system-site-packages, so that
there's no risk of interaction with any Python libraries that might
somehow get subsequently installed on the system.
-- 
Jeremy Stanley