[Catalog-sig] start on static generation, and caching - apache config.
Phillip J. Eby
pje at telecommunity.com
Fri Jul 13 01:43:04 CEST 2007
At 07:14 PM 7/12/2007 -0400, Jim Fulton wrote:
>On Jul 12, 2007, at 5:09 PM, Martin v. Löwis wrote:
> >> 2. It's important to reduce the number of round trips.
> > A colleague today suggested that the best way to reduce round trips
> > is to give each machine a local copy of the index, the same way
> > Debian apt works: you do 'apt-get update', and then have a local
> > copy of the catalog that you can build against. No roundtrips
> > at all (except for the one to update the local catalog), for the
> > expense of being out of date if you don't manually update the
> > catalog.
>Yup. This might be a really nice way to go. It would be especially
>nice if a client could contact PyPI and ask for new data since a
>given time. I imagine that this request could be as cheap as the
>requests we have now, unless a client was very out of date.
Such a query could simply consist of which packages had been updated,
and the data could then be cleared from the local cache.
The downside to this approach is that it's not any faster for
anything you've never downloaded before.
So, I'm not really sure how to create a quality user experience with
edge caching alone. It seems to me that geographically localized
mirrors are needed to provide infrequent users and new users with
good performance. And presumably, the commercial users who are
having issues now, want their users as well as their developers to
have good performance.
(Personally, I find it extremely irritating every time the "yum"
package manager makes me wait for it to download a bunch of
repository data that isn't necessarily even related to what I just
asked it to do.)
More information about the Catalog-SIG