[pypy-dev] Benchmarking PyPy performance on real-world Django app

[Igor Katson]

When I first started benchmarking one of my Django sites, 
http://trip-travel.ru/ (using postgres driver pypq),
I was disappointed by the results. PyPy was visually much slower than 
cPython (I just looked at how the site loads in the browser)

But today I got some incredible results, I finally made PyPy to work 
faster than cPython, and found out that it got faster after loading the 
page several hundred times with "ab" or "siege"

Here a the results, of mean response time by querying the home page with 
apache's "ab" (cPython 2.7.2, Django 1.3, PyPy 1.6.0), served with 
cherrypy wsgi server:

After 10 requests (excluding the first request):
cPython - 163.529 ms
PyPy - 460.879 ms

50 request more:
cPython - 168.539
PyPy - 249.850

100 requests more:
cPython - 166.278 ms
PyPy - 131.104

100 requests more:
cPython - 165.820
PyPy - 115.446

300 requests more:
cPython - 165.543
PyPy - 107.636

300 requests more:
cPython - 166.425
PyPy - 103.065

As we can see, the JIT needs much time to warm up, but when it does, the 
result is pretty noticeable.
By the way, with psycopg2, the site responds for 155.766 ms in average 
(only 10 ms faster), so using PyPy with Django makes much sense for me.

As for now, pypy cannot run with uWSGI, which I use in production, but 
maybe i'll switch to PyPy for production deployments if "PyPy + PyPQ + 
Some pure python WSGI server" suite will outperform (uWSGI + cPython + 
psycopg2). Though, the need to load the page 500 times after each server 
reload is not comfortable.
<http://trip-travel.ru/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20111007/1cbf4a76/attachment.html>