[Catalog-sig] pre-PEP: transition to release-file hosting at pypi site

[PJ Eby]

On Tue, Mar 12, 2013 at 12:29 PM, Jacob Kaplan-Moss <jacob at jacobian.org> wrote:
> On Tue, Mar 12, 2013 at 11:19 AM, M.-A. Lemburg <mal at egenix.com> wrote:
>> So let's do this carefully and find a good solution before
>> jumping to conclusions.
>
> Completely agreed; rushing is a bad idea.
>
> But so is not starting. What I'm seeing — as a total outsider, a user
> of these tools, not someone who creates them — is that a bunch of
> people (Holger, Donald, Richard, the pip maintainers, etc.) have the
> beginnings of a solution ready to go *right now*, and I want to
> capture that energy and enthusiasm before it evaporates.
>
> This isn't an academic situation; I've seen companies decline to adopt
> Python over this exact security issue.

Nobody told them about how to configure a restricted, site-wide
default --allow-hosts setting?   (
http://peak.telecommunity.com/DevCenter/EasyInstall#restricting-downloads-with-allow-hosts
and http://docs.python.org/2/install/index.html#location-and-names-of-config-files
)

(FWIW, --allow-hosts was added in setuptools 0.6a6 -- *years* before
the distribute fork or the existence of pip, and pip offers the same
option.)

I've already agreed to change setuptools to default this option to
only allow downloads from the same host as its index URL, in a future
release.  (i.e. to default --allow-hosts to the host of the
--index-url option), and I support the removing of rel="" spidering
from PyPI (which will significantly mitigate the immediate speed and
security issues).  Heck, I've been the one who'se repeatedly proposed
various ways of cutting back or removing rel="" attributes from the
/simple index.

The result of these two changes will actually have the same net effect
that people are being asking for here: you'll only be able to download
stuff hosted on PyPI, unless you explicitly override the --allow-hosts
to get a wider range of packages.

Already today, when a URL is blocked by --allow-hosts, it's announced
as part of easy_install's output, so you can see exactly how much
wider you need to extend your trust for the download to succeed.

The *only* thing I object to is removing the ability for people to
*choose* their own levels of trust.

And I have not yet seen an argument that justifies removing people's
ability to *choose* to be more inclusive in their downloads.

And I've put multiple compromise proposals out there to begin
mitigating the problem *now* (i.e. for non-updated versions of
setuptools), and every time, the objection is, "no, we need to ban it
all now, no discussion, no re-evaluation, no personal choice, everyone
must do as we say, no argument".

And I don't understand that, at all.