[Distutils] What to do about the PyPI mirrors

[Nick Coghlan]

On 6 August 2013 16:59, Christian Theune <ct at gocept.com> wrote:
> Hi,
>
> Thanks for all the feedback, I'll calm down a bit and ponder some more structured reply.
>
> However, you're responding to the technicalities. I didn't see any consideration to the user pain. It seems irrelevant. Almost like arguing with the TSA about taking off your shoes.

User pain is the only reason for not making the change tomorrow.
People need time to adjust, or to propose alternative solutions.

> f.pypi.python.org is going to go away. And *everyone* using it needs to change it. Manually - or else.

Delegating subdomains of python.org without a contractual relationship
in place was a fundamental mistake. It should never have happened. We
can either admit "We screwed up and set up a seriously flawed
mirroring system" and take steps to fix it, or we can leave the
HTTP-only mirrors open as a security hole forever.

One means by which I could see an f.pypi.python.org DNS record being
left in place indefinitely is if the TUF folks are able to come up
with a scheme for offering end-to-end security for the *existing* PyPI
metadata, *and* the TUF metadata is mirrored by bandersnatch *and* the
TUF client side integrity checks are invoked by pip. In that case, the
security argument regarding the lack of TLS on the subdomains would be
rendered moot, and the backwards compatibility argument for keeping it
active would win.

Another potential alternative might be for Gocept to approach the PSF
about getting an SSL certificate for that domain, ensuring pip and
setuptools both support HSTS, and then switching that mirror over to
using HSTS (so even configurations hardcoded to use
http://f.pypi.python.org will still get a validated secure
connection).

Both of those approaches would close the security hole, while leaving
the domain in place. If upgrading pip and easy_install clients is a
more acceptable solution than updating affected configurations to use
a different domain name, then these are certainly options we should
discuss.

The only option which I consider completely out of the question is
leaving f.pypi.python.org (or any other *.pypi.python.org subdomain)
in place indefinitely as an insecure HTTP-only endpoint.

> Other communities, like the Linux distributions, are doing simple, file-based stuff for ages. They did not learn from us, and AFAICT we didn't learn from them?

This case *is* a matter of us learning from other mirroring systems:
none of them are based on delegating subdomains to third parties,
they're all based on lists of mirror URLs, and some mechanism for
retrieving that list. However, as long as the flawed way remains
blessed as the official mirroring network, it's difficult for an
alternative model to gain any traction.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia