[Catalog-sig] Deprecate External Links

[Donald Stufft]

On Wednesday, February 27, 2013 at 3:16 PM, holger krekel wrote:
> On Wed, Feb 27, 2013 at 14:49 -0500, Monty Taylor wrote:
> > On 02/27/2013 02:47 PM, Aaron Meurer wrote:
> > > On Wed, Feb 27, 2013 at 11:37 AM, holger krekel <holger at merlinux.eu (mailto:holger at merlinux.eu)> wrote:
> > > > On Wed, Feb 27, 2013 at 19:34 +0100, Lennart Regebro wrote:
> > > > > On Wed, Feb 27, 2013 at 5:34 PM, M.-A. Lemburg <mal at egenix.com (mailto:mal at egenix.com)> wrote:
> > > > > > I'm not saying that it's not a good idea to host packages on PyPI,
> > > > > > but forcing the community into doing this is not a good idea.
> > > > > > 
> > > > > 
> > > > > 
> > > > > I still don't understand why not. The only reasons I've seen are
> > > > > "Because they don't want to" or "because they don't trust PyPI". And
> > > > > in the latter case I'm assuming they wouldn't use PyPI at all.
> > > > > 
> > > > > And of course, nobody is forcing anyone, just like nobody is forcing
> > > > > you to use PyPI. :-)
> > > > > 
> > > > 
> > > > 
> > > > I understood there is the idea to disable external links within a couple
> > > > of months. That does break backward compatibility in a considerable way.
> > > > 
> > > > holger
> > > 
> > > But wouldn't this only be a change in pip/easy_install, not PyPI
> > > itself? I suppose you could explicitly break the external links by
> > > having them point to nothing if you are worried about the security or
> > > if it's some performance issue (that would indeed be a bad
> > > compatibility break, in case people are using those for other
> > > purposes). Otherwise, if it's a problem, then just use the old
> > > version of pip.
> > > 
> > 
> > 
> > If we don't remove the feature from pypi itself, then it won't help the
> > folks for whom its a problem, because there will be no incentive for the
> > folks hosting their software that way to actually upload their stuff to
> > PyPI - which means that client-side disabling of external_links is
> > fairly likely to never be usable.
> > 
> 
> 
> I can see it's tempting to just try to "force" everyone to upload
> their stuff to pypi.python.org (http://pypi.python.org). I am very skeptical about this approach.
> 
> There already is a high frustration with the packaging ecology
> in Python. When we remove external links on the server side, installs
> for many people and companies are going to break, no matter what. And
> they would have no client-side switch anymore to make things working.
> Requiring to use older setuptools/pip versions would not help because
> the server information is gone. That'd just increase frustration.
> 
> 

"Locking the bank door will frustrate people when they have to ask the teller
for their money instead of walking into the vault and grabbing it themselves". 

I'm not asking for this to be shutoff immediately, it will be phased,
particularly so project maintainers can be made aware that it's
going away and can upload versions to PyPI to prevent this kind of
wide spread breakage. Particularly the first phase I outlined for
PyPI was to disable _new_ links from being added to the /simple/
pages and keeping the old around. So that _old_ releases still work
for now, but _new_ ones do not.

Further more I do have issues for both pip and buildout to implement
this there as a warning at first, and then move to disabling them by default
eventually. The goal again to make the fact it's going away a *known*
fact to give maintainers time to upload packages.

There will be some breakage, particularly with unmaintained  software (but
if it's truly unmaintained then it's likely to break at anytime when the
external host goes away).
> 
> So at the very least using external links needs to be a client-side
> installer choice for a long while and the server needs to offer
> the according information.
> 
> I'd generally prefer to think hard about ways to improve the situation
> without breaking things. Putting simple/ and packaging serving on a CDN
> is one such step and a good idea i think. Establishing a
> signing/verification mechanism is another. Refining py2/py3 dependency
> discovery yet another good one.
> 
> 

Sometimes you need to break things. The goal is to do it with ample
warning and migration time so that people have a chance to move
to the new way of doing things.

Again, I am not suggesting we delete all external links immediately, just
disable new ones. Removing old ones will come later.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130227/72d5a16c/attachment.html>