[Catalog-sig] Deprecate External Links

[PJ Eby]

On Thu, Feb 28, 2013 at 4:28 AM, Lennart Regebro <regebro at gmail.com> wrote:
> My suggestions to move forward on this issue is as follows:
>
> 1. New versions of pip and distribute are released that will start
> warning if they download distributions that are not from PyPI, unless
> explicitly given a URL to download.

I can't speak to pip, but since the relevant bits of distribute are
95% the same as setuptools, I think I can say that it will have the
same technical issues, and that warning based on lack of an
--allow-hosts will be both simpler to implement and easier to make
secure.  The internal architecture of easy_install is such that it
does not really know where a URL came from; there isn't really a
difference between a command-line argument and a dependency_links file
that came from somebody else's code.  A configuration-specified
allow-hosts can also prevent installs that are run from code instead
of directly from the command line (e.g. setup_requires or
tests_require).

That being said, what you propose *could* perhaps be done, but it
would require a fair amount of code review to track every possible
path of URLs getting to the fetch system, and the fetch system already
has URL filtering using allow-hosts.  Anyway, the path of least
resistance is allow-hosts, and anybody who cares about not downloading
external URLs or wants other people not to download them should be out
there promoting how to set a sitewide allow-hosts default to accept
only PyPI.  With a suitably restricted allow-hosts, neither setuptools
nor distribute will follow or download external links, it'll just say
it's not following them due to allow-hosts.

For that matter, if pip has an allow-hosts and reads it from the
distutils config files' easy_install sections, that'll *also* fix pip.
 (But I don't know if it goes that far w/easy_install backward
compatibility; I've never actually used it.)


> 2. After a pre-determined period (6 months?) new versions are again
> released that no longer download from external sites, unless a
> parameter is added. We still warn when the parameter is added that
> this feature will go away.

I'd suggest that this be simply making the default --allow-hosts point to PyPI.

(To be clear, at least on the setuptools side, the feature itself will
not be going away, at least not until the new infrastructure arrives.
When the new tools are ready, setuptools can just fade off into the
sunset.)


> 1. As far as I can tell, there is no way to ask PyPI what version of
> the API it's running. Is this correct? If so that should be added. For
> the /simple/ API we could stick a version header as metadata in the
> header, maybe?
>
> 2. We determine a version number that will break backwards
> compatibility, is every major version increase.
>
> 3. New versions of pip and distribute will check these version numbers
> and warn (but not fail) if the major version increases, noting that
> it's time to upgrade.

I think we should do something more like what MAL is proposing, which
means that the current "API" can disappear altogether when the new
tools arrive.