[Catalog-sig] start on static generation, and caching - apache config.

["Martin v. Löwis"]

> I wonder what times ab against  http://www.python.org/pypi/ZODB3 from  
> inside the python.org network would give.

I just measured it. 1000 requests take 17s using urllib, giving 60
request per second.

> I wonder if it would help much to make multiple HTTP requests in the  
> same connection.  This might be something to look at in setuptools  
> and/or xmlrpclib.

Only for remote connections, due to the round-trips required for
TCP handshake. Locally, Apache opens a new connection to the FCGI
servers per requests (using the farmer-worker pattern).

> 1. Since many people will be far away from PyPI, I think our long- 
> term plan should encompass geographic mirrors.  It's good that the  
> server is spending a small amount of time, but it still takes *me* a  
> long time to get data.

Ok. I am, in general, skeptical about mirroring. However, if it
makes people happy, feel free to implement it.

A number of issues should be considered, of course:
- there should be a way to get authoritative answers somehow, preferably
  from mirrors, but, if necessary, from the main site
- I really wish to collect download counters across mirrors. "Official"
  mirrors should be obliged to report download statistics once a day
  or so.

> 2. It's important to reduce the number of round trips.

A colleague today suggested that the best way to reduce round trips
is to give each machine a local copy of the index, the same way
Debian apt works: you do 'apt-get update', and then have a local
copy of the catalog that you can build against. No roundtrips
at all (except for the one to update the local catalog), for the
expense of being out of date if you don't manually update the
catalog.

Regards,
Martin