[Distutils] PEP449 - Removal of the PyPI Mirror Auto Discovery and Naming Scheme

[Donald Stufft]

On Aug 27, 2013, at 7:10 AM, holger krekel <holger at merlinux.eu> wrote:

> Being one of the people who wanted to but didn't feedback (still in vacation,
> writing from a camping place with ssh/mutt and lousy connectivity FWIW): 

Ah! I hope you're having fun :)

> 
> - the PEP claims that PEP381 mirroring protocol continues to exist.
>  But are the statements in http://www.python.org/dev/peps/pep-0381/#statistics-page still valid, i.e. does pypi.python.org still crawl mirrors for
>  statistics when the PEP449-DNS removal happens?  Also PEP381 has
>  seen some modifications and enhancements before and after the CDN
>  introduction.

PyPI hasn't crawled mirrors for statistics for 3 months? or so. Ever since
download counts first got shut off and it has never been restored. Personally
I have no plans to restore it. It caused needless complication and the
download counts from the mirrors aren't that important IMO. The download
counts are already inaccurate and are primarily useful as a form of
relative comparison so the additional numbers do not aid much in the
form of relativeness only in absolute counts (which as stated are already
widely inaccurate).

Perhaps PEP381 should be updates to take into account the new abilities
added for mirrors that have happened recently (the Serials, the Headers etc).
It also probably makes sense to update it since PyPI is no longer fetching
statistics from the mirrors. I don't think we need a new mirror PEP though
as the mirroring protocol is mostly the same and the enhancements exist
primarily as a means of getting a more accurate mirror.

I *do* have plans down the road to introduce a new mirroring protocol
but that is a ways out still as there are other things higher up on my todo
list.

> 
> - relatedly, I'd suggest to clarify that this PEP does at least not preclude 
>  further PEPs or attempts to introduce other means than DNS to manage 
>  PyPI mirrors (one where mirror availability is stored at and queried via 
>  a python.org address).  Ideally, it should already incorporate a
>  procedure to register mirrors and to list them at a web page.

I don't see this PEP as precluding anything else. Currently it points to
http://pypi-mirrors.org/ as the place to locate new mirrors from in a manual
fashion. I'm not too concerned with an automatic discovery protocol since
the only installer as far as I'm aware that even used the existing one was
pip which is removing that support in 1.5 anyways.

That being said I'm not opposed to a new PEP introducing a different scheme
but I probably won't be the architect	 of it and I can make a small update to
it that it doesn't preclude further PEPs if that would make people feel more
comfortable.

> 
> - maybe a "future work" section could list these issues.
> 
> I guess one underlying question is how much we want to rely on the CDN
> mid/long-term.  It's introduction was not discussed in a PEP but it
> is mentioned e.g. in PEP449 as a reason to shutdown mirror management
> infrastructure.

Personally I see the CDN as the best option for the bulk of people wanting
to install from a *public* mirror. There are of course situations where you
might want to install from a different public mirror (China being a big one). I
see mirrors mostly being useful for smaller use cases now. However I have
no plans or desire to make the public mirrors go away other than the existing
DNS names (and only then because of security concerns).

> 
> That all being said, i am otherwise ok with PEP449 as DNS seems indeed 
> the wrong way to handle mirror management.

Awesome good to hear.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130827/b7ef7a72/attachment.sig>