On Oct 7, 2014, at 7:03 AM, holger krekel
wrote: On Tue, Oct 07, 2014 at 11:40 +0100, Paul Moore wrote:
On 7 October 2014 11:09, holger krekel
wrote: Well, the main benefit of PEP438 was that it removed random crawling for some 90% of the packages on the package index, speeding up and making installs more reliable. And it did that without breaking backward compatibility.
The setuptools index page is 1.4MB in size. Most of that can be ignored, but it still has to be downloaded and parsed. Whether the data that setuptools includes in its long_description is useful is arguable, but irrelevant - the fact is that as things stand, it is there and it causes issues.
PEP 470 would result in all of the unneeded entries in the simple index for setuptools being removed, which avoids the need for client tools (and I'm not talking just about pip here, but also about one-off scripts, which is the sort of thing I write a lot) to trawl through all of that data. And it does so without the setuptools project having to change how it writes its PyPI page (i.e., the project long_description). Arguably, that's equally a way of avoiding breaking backward compatibility...
The second could be done without breakage alltogether i think: at one time all external urls are auto-registered as external indexes and they are presented on the simple page with some meta information that does not confuse older pips/easy_installs. Newer pips/easy_installs can then provide nice error messages. Older pips can continue to use the PEP438 options. And easy install can continue to work.
Setuptools has 255 internal links to files hosted on PyPI. And about 11,000 other links. (I just checked that 3 times, as I couldn't believe it, but it *seems* to be right :-(). Removing duplicates, 337 unique links. Are you suggesting pip presents all of those as possible external indexes?
No, i effectively suggest that PyPI would present just 2 index links, those which currently are attributed as rel={download,homepage}. Those two index links would be put into the new "extra indexes field" on pypi with a note like "the following indexes were extracted from old release data" which newer pip versions could present to the user. For older pip/easy_installs things would just continue to work but they'd get a shorter setuptools simple page.
I am not opposed to moving the rel={download,homepage} automatically from link to metatag, I am opposed to leaving them in place in an attempt to prioritize backwards compatibility over safety. The only thing I have against automatically translating the old links to the new external hosting metadata is that it’s going to be a lot of noise for authors who won’t know which links are the correct links to use. --- Donald Stufft PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA