[Catalog-sig] start on static generation, and caching - apache config.

[Phillip J. Eby]

At 07:14 PM 7/12/2007 -0400, Jim Fulton wrote:

>On Jul 12, 2007, at 5:09 PM, Martin v. Löwis wrote:
> >> 2. It's important to reduce the number of round trips.
> >
> > A colleague today suggested that the best way to reduce round trips
> > is to give each machine a local copy of the index, the same way
> > Debian apt works: you do 'apt-get update', and then have a local
> > copy of the catalog that you can build against. No roundtrips
> > at all (except for the one to update the local catalog), for the
> > expense of being out of date if you don't manually update the
> > catalog.
>
>Yup. This might be a really nice way to go. It would be especially
>nice if a client could contact PyPI and ask for new data since a
>given time.  I imagine that this request could be as cheap as the
>requests we have now, unless a client was very out of date.

Such a query could simply consist of which packages had been updated, 
and the data could then be cleared from the local cache.

The downside to this approach is that it's not any faster for 
anything you've never downloaded before.

So, I'm not really sure how to create a quality user experience with 
edge caching alone.  It seems to me that geographically localized 
mirrors are needed to provide infrequent users and new users with 
good performance.  And presumably, the commercial users who are 
having issues now, want their users as well as their developers to 
have good performance.

(Personally, I find it extremely irritating every time the "yum" 
package manager makes me wait for it to download a bunch of 
repository data that isn't necessarily even related to what I just 
asked it to do.)