[Catalog-sig] V4 Pre-PEP: transition to release-file hosting on PYPI

Carl Meyer carl at oddbird.net
Fri Mar 15 18:39:58 CET 2013

On 03/15/2013 10:51 AM, PJ Eby wrote:
> Giving a blanket pass to all external links doesn't seem like
> such a good idea to me, 

This is a very good point, and it should be made clearer in the PEP that
we don't recommend a single blanket option to allow all external links,
but an option (like allow-hosts) that lets you specify with more
granularity which external links to use. I think perhaps rel="external"
confuses this point; the real purpose of the rel tags is just so that
rel="internal" can be considered "part of the index."

FWIW I think it would be just as reasonable UI for a hypothetical tool
to let you say "I want to trust external links for the Foo project"
rather than "I want to trust external links to djangoproject.com" and
avoid host-comparison altogether. IOW, I don't think "hostname" is
inherently a better or safer indicator of trust than "project name";
hosts can change ownership at least as easily and silently as PyPI
projects! So I don't think the PEP should require all installer tools to
choose trust-by-hostname (which would be implied by removing the rel tags).

> nor does allowing the index to define what
> hosts the client should trust.   

I'm not sure about this. By using an index at all, you are trusting that
index to provide whatever level of
reliability/stability/security/whatever you expect from it. Allowing the
index itself to specify that it keeps its files on a different host in a
way that is transparent to the user seems like a natural extension of
this trust that doesn't harm anything and aids usability greatly. (Cases
where the index is lying to you definitely fall outside the scope of
what this PEP is aiming to help with.)

As for the internal ones, I'm not
> sure why we can't at least make a subdomain requirement, or have users
> explicitly add a PyPI CDN to their configured --allow-hosts.

Even a subdomain requirement can make a CDN more difficult/expensive to
implement. And once you go beyond simple host-equality comparisons and
into subdomain-equivalence I'm wary of the added implementation
complexity we're asking of every installer tool, and the potential for
subtle differences in implementation. This seems to me like a worse can
of worms than rel-parsing.

> To try to put it another way: there should be one, and preferably only
> one, obvious way to specify where you get downloads from.  That way in
> easy_install is currently --allow-hosts.  Adding new options that
> interact and overlap with that looks like bad UI design to me,
> increasing the possibility of user confusion.

Like Donald says, I don't see any problem with you choosing to keep
allow-hosts as the only user-facing option for easy_install. It would be
up to you whether you also want to use rel="internal" as a hint for
implicitly (perhaps with warning) adding to --allow-hosts, to allow
better compatibility with indexes that use a different host for
file-hosting (it's possible that even PyPI itself may move into this
category, I haven't been following the CDN discussions carefully).

PyPI wouldn't be enforcing a UI on you here, just providing metadata
that you can use as you wish. I do think the internal/external
distinction is meaningful and unambiguous metadata that the index is
able to provide, and there's no reason for the index to withhold it.
(That distinction is not new in this version of the PEP, either, it's
just made via rel tags now instead of via a separate index.)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/catalog-sig/attachments/20130315/efa2228e/attachment.pgp>

More information about the Catalog-SIG mailing list