[Catalog-sig] start on static generation, and caching - apache config.

Jim Fulton jim at zope.com
Wed Jul 11 16:03:33 CEST 2007

On Jul 11, 2007, at 12:04 AM, richardjones at optusnet.com.au wrote:

> Stephen Waterbury <waterbug at pangalactic.us> wrote:
>> Martin v. Löwis wrote:
>>> [Jim Fulton wrote:]
>>>> Maybe others can chime in.
>>> That's also my concern. Nobody else is complaining; AFAICT, there
>>> is just one unhappy user of PyPI.
>> I'm not happy with PyPI's performance either.
>> Probably many users are like me:  I thought it was
>> common knowledge that the performance of PyPI was bad, but
>> I didn't want to complain when it appeared that people were
>> working on improvements.
> It has been slow in the past, but Martin has done some great work  
> speeding it up in the last few days.

Yup. Much thanks Martin!

> If it's still slow, please report when you noticed and what you  
> were trying to do.

Let's look at the new-improved times.  Right now ~14:00UTC July 11:

   http://www.python.org/ZODB3 takes about .3 seconds (median)(means  
is higher)
   http://www.python.org/ZODB3/3.8.0b2 also takes about .3 seconds
   http://www.python.org/pypi/ takes aabout 6 seconds (median)

For the sake of argument, let's ignore http://www.python.org/pypi/.

The .3-second times per request is *much* better than we had before  
(I assume), but it's *not fast enough*.  The demand on the package  
index used by setuptools is going to increase substantially.  Even if  
setuptools only made a single request per package, .3 seconds per  
request is too slow.  Given the current structure of the index,  
setuptools has to make a request for the package and a request per  
release.  For ZODB, this means about 12 requests, or more than 3  
seconds.  Of course, this will increase over time, as more releases  
are made.

The progress Martin has made has (I assume and hope) greatly  
increased the reliability and performance of PYPI.  This is very  
important and much appreciated.  It is not enough in the long (or, I  
suspect medium) term.


Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org

More information about the Catalog-SIG mailing list