[Catalog-sig] V2 pre-PEP: transitioning to release file hosting on PYPI

[Carl Meyer]

Hi Holger,

I am confused about the discrepancy between the title of this pre-PEP
("transition to release file hosting on PyPI") and the contents of the
PEP, which describe a transition to not crawling _HTML pages_ on
external sites looking for distribution download links. These are not
the same thing at all.

Current installer tools will only crawl external HTML pages if they are
rel="download" or rel="homepage", but they will use any link they find
in the simple index (regardless of rel attr) if the target of the link
appears to be a distribution file (as determined by filename
pattern-matching or #egg fragment).

At the end of the process you describe, if all packages migrate to
"nocrawl", the rel-link HTML spidering will no longer happen. This is a
good first step: it will speed up installation somewhat, and reduce the
frustration of some package owners when installers find files linked
from their project homepage that they never intended for automated
installation. But installers will still find and download release
packages that are not hosted on PyPI, if those package files are linked
directly in the simple index. This is still surprising behavior to many
new Python users, and still carries the security and reliability
concerns that this PEP claims to address.

I'm honestly not sure whether the title or the content more accurately
reflects the intent of this PEP; depending which it is, I suggest one of
the following:

1) Add to the PEP a description of a further step in the migration
process, which actually does transition away from automated installation
of non-PyPI-hosted release files (as the default behavior of
installation tools); or

2) Change the title of the PEP to something like "Transitioning away
from non-PyPI HTML crawling" and add a paragraph to the PEP clarifying
that this PEP does not address the issue of actual release files hosted
off-PyPI.

Carl