[Catalog-sig] start on static generation, and caching - apache config.
jim at zope.com
Thu Jul 12 12:34:19 CEST 2007
On Jul 12, 2007, at 1:11 AM, Martin v. Löwis wrote:
>> ATM, from my machine, xml-rpc requests to PyPI are taking about .27
>> seconds. This is only a little less than regular page requests.
>> the current API, It would require at best 3 requests to get all of
>> distribution URLs. Presumably, with a change to the API, we could
>> this down to one request, but that's still a long time given the
>> I expect on PyPI in the future.
> You seem to assume that if you see a round trip time of .27 seconds,
> that then PyPI could only do 3 requests per second. That is not so.
Yeah, it occurred to me on my way home that a substantial part of the
time might be due to distance.
I wonder what times ab against http://www.python.org/pypi/ZODB3 from
inside the python.org network would give.
I wonder if it would help much to make multiple HTTP requests in the
same connection. This might be something to look at in setuptools
> So it can currently do 130 XML-RPC requests per second, to
> a single client. Inverting it, a request takes 0.0076s,
> which is a lot less than 0.27s.
Cool. That's much better. Thanks for trying this.
OTOH, this points up a couple things:
1. Since many people will be far away from PyPI, I think our long-
term plan should encompass geographic mirrors. It's good that the
server is spending a small amount of time, but it still takes *me* a
long time to get data.
2. It's important to reduce the number of round trips.
I'm still opposed to using XML-RPC because:
- It's harder to mirror, and
- It's still slower than static pages.
Note that after our discussion, I'm equally against the current
approach of parsing a human interface. I still think it makes a lot
more sense to have a tailored interface for setuptools.
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Catalog-SIG