[Catalog-sig] Cheese Shop performance
richardjones at optushome.com.au
Mon Apr 9 02:01:55 CEST 2007
I've been quiet on the latest happenings because I've have my attention wholly
consumed by PyWeek and gettng a new job.
The performance issues are related to spiders crawling the wiki and the Cheese
Shop "browse" interface. The PyPI code attempts to prevent bots from
accessing that interface, but that requires that we know user-agent strings
in advance. And people are writing newer, dumber bots all the time :(
There's a very good chance that I'll be able to do some work over the next few
weeks on the baking ideas that Jim Fulton proposed at PyCon. This would make
most of the user interface (except the browse) static where it currently goes
through CGI for all requests. In that situation I'm not sure we'll need to
have mirrors, though it would certainly help if they were needed.
More information about the Catalog-sig