[Catalog-sig] Proposal to Reverse Ordering of Scraped Links in PyPI

P.J. Eby pje at telecommunity.com
Mon Jul 26 01:07:30 CEST 2010


At 11:53 PM 7/25/2010 +0100, Michael Foord wrote:
>Heh pip, distribute and setuptools *really* shouldn't be scraping 
>pypi - at least except as a last resort. pypi has an xml-rpc (and a 
>fledgling json api) which should be used in preference to scraping.

This is done to allow users to implement their own "PyPI clone" using 
nothing more than a static webserver or webpage.  XML-RPC would 
require a dynamic server.


>This *already* causes problems for pypi maintenance.

For some time now, easy_install uses the '/simple' index 
(specifically intended for automated tools' consumption), rather than 
the human-oriented pages.  Among other benefits, the /simple index 
can be served or mirrored statically, rather than being generated 
anew on each hit.



More information about the Catalog-SIG mailing list