[Distutils] PyPI is a sick sick hoarder

[Robert Collins]

On 16 May 2015 at 07:19, Donald Stufft <donald at stufft.io> wrote:

> There have been a handful of projects which would only keep the latest N
> versions uploaded to PyPI. I know this primarily because it has caused
> people a decent amount of pain over time. It’s common for deployments people
> have to use a requirements.txt file like ``foo==1.0`` and to just continue
> to pull from PyPI. Deleting the old files breaks anyone doing that, so it would
> require either having people bundle their deps in their repositories or
> some way to get at those old versions. Personally I think that we shouldn’t
> go deleting the old versions or encouraging people to do that.

I think 'most recent only' is too much. Most upstreams will support
more than one release. Like - I don't care what testtools release you
use.

OTOH, every version with distinct dependencies becomes a very
expensive liability to the ecosystem here. It's beyond human scale,
and well in the territory of argh wtf the universe is burning around
me and my tardis has run out of power.

I'm sure we can provide an escape hatch in pip (and I'm going to do
that in my branch soon - offering simple 'error on conflict' and 'use
first seen specifier only' strategies) while folk work on different
heuristics - the actual resolver is only ~100 LOC in my branch today -
the rest is refactoring (that can be made better and I plan to do so
before suggesting we merge it).

But a significant contributing factor is the O of the problem, and we
can do something about that. I don't know what exactly, and I think
we're going to need to have our creative caps firmly on to come up
with something meeting the broad needs of the ecosystem: which
includes pip Just Working.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud