[pypy-dev] djangobench performance

[Fenn Bailey]

Hi all,

As an experiment, I thought I'd test JKM's djangobench (
https://github.com/jacobian/djangobench) under pypy as a way of determining
a (hopefully) more useful benchmark than the template-only "django"
benchmark that's standard on speed.pypy.org and also to get an idea as to
whether switching to pypy for production django apps could (currently) be a
good idea.

djangobench is designed to fairly comprehensively compare the performance of
different aspects of differing versions of django in an effort to detect
performance degradation/regression/etc.

It's based on perf.py from the unladen swallow project, so it was fairly
easy to crudely hack up to instead compare a single django version running
under cpython 2.6 vs pypy 1.6.

---
$ python -V
Python 2.6.5
$ pypy -V
Python 2.7.1 (d8ac7d23d3ec, Aug 17 2011, 11:51:19)
[PyPy 1.6.0 with GCC 4.4.3]
---

The results were a little surprising (and not in a good way):
http://pastie.org/2463906

Based on the highly degraded performance (>2 orders of magnitude in some
cases) I'm guessing there's some sort of issue in the way I'm benchmarking
things.

Code can be found here: https://github.com/fennb/djangobench

Environment is ubuntu 10.04 64bit running in a VM on a macbook pro. cpython
was the current ubuntu binary package, pypy was 1.6 precompiled binary from
pypy.org. It's quite possible memory size issues may have impacted some of
the benchmarks (but not all).

Any ideas as to why the performance drop-off would be so significant?

Cheers,

  Fenn.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110901/a53fb7a8/attachment-0001.html>