[Distutils] PEP 470 discussion, part 3

[Richard Jones]

I have been mulling over PEP 470 for some time without having the time to
truly dedicate to addressing it. I believe I'm up to date with its contents
and the (quite significant, and detailed) discussion around it.

To summarise my understanding, PEP 470 proposes to remove the current link
spidering (pypi-scrape, pypi-scrape-crawl) while retaining explicit hosting
(pypi-explicit). I believe it retains the explicit links to external
hosting provided by pypi-explicit.

The reason given for this change is the current bad user experience around
the --allow-external and --allow-unverified options to pip install. That
is, that users currently attempt to install a non-pypi-explicit package and
the result is an obscure error message.

I believe the current PEP addresses the significant usability issues around
this by swapping them for other usability issues. In fact, I believe it
will make matters worse with potential confusion about which index hosts
what, potential masking of release files or even, in the worst scenario,
potential spoofing of release files by indexes out of the control of
project owners.

I would like us to consider instead working on the usability of the
existing workflow, by rather than throwing an error, we start a dialog with
the user:

$ pip install PIL
Downloading/unpacking PIL
  PIL is hosted externally to PyPI. Do you still wish to download it? [Y/n]
y
  PIL has no checksum. Are you sure you wish to download it? [Y/n] y
Downloading/unpacking PIL
  Downloading PIL-1.1.7.tar.gz (506kB): 506kB downloaded
...

Obviously this would require scraping the site, but given that this
interaction would only happen for a very small fraction of projects (those
for which no download is located), the overall performance hit is
negligible. The PEP currently states that this would be a "massive
performance hit" for reasons I don't understand.

The two prompts could be made automatic "y" responses for tools using the
existing --allow-external and --allow-unverified flags.

I also note that PEP 470 says "PEP 438 proposed a system of classifying
file links as either internal, external, or unsafe", whereas PEP 438 has no
mention of "unsafe". This leads "unsafe" to never actually be defined
anywhere that I can see.

Finally, the Rejected Proposals section of the PEP appears to have a couple
of justifications for rejection which have nothing whatsoever to do with
the Rationale ("PyPI is fronted by a globally distributed CDN...", "PyPI
supports mirroring...") As Holger has already indicated, that second one is
going to have a heck of a time dealing with PEP 470 changes at least in the
devpi case.

 "PyPI has monitoring and an on-call rotation of sysadmins..." would be
solved through improving the failure message reported to the user as
discussed above.


     Richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20140723/d57b7eb1/attachment.html>