[pypy-dev] Quick "state of performance" analysis

[Miquel Torres]

Hi all!

I have implemented a small new feature for the speed center. In the
Overview, you are now able to select to which results you wish to compare
the performance of the selected interpreter. That allows a couple of
interesting investigations to be made (bear with me if I say obvious
things):

1 - http://speed.pypy.org/overview/?interpreter=3)
We already knew that there are 5 benchmarks where pypy-c without the jit is
more than twice times slower than cpython: slowspitfire, twisted_iteration,
twisted_tcp, html5lib and worst of all spambayes.

2 - http://speed.pypy.org/overview/
The jit makes up for some of them, only twisted_tcp, slowspitfire and
spambayes remain as cases where cpython is much faster.

3 - http://speed.pypy.org/overview/?baseline=4
We can now compare pypy-c-jit with cpython with psyco acceleration. The
first thing that stands out, is that pypy is much slower... exactly in the
same cases where pypy-c without the jit is slower than cpython (no
surprise). The second one, is that apart from those 5 cases where the jit is
held back by bad pypy-c baseline performance, pypy-c-jit is already much
better than psyco, about twice as fast!

4 - http://speed.pypy.org/overview/?baseline=3
Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can
confirm that the jit accelerates *everything* that we currenly measure, and
most by a very good factor. The 3 slow benchmarks from comparison 2 are here
last, the are accelerated the least by the jit.

So to me the conclusion is that the jit has good enough performance for a
first version, and that only poor pypy-c performance in some cases is
holding the pypy project from becoming a better option than cpython.
Wouldn't it be a good aim to make it a priority of the next (1.3?) release
to improve on that? ;-)

Apart from it making pypy more "ready", there is a problem if that is not
addressed while further jit development continues. An application is only so
fast as its slowest part, so even if the jit accelerates some cases
100-fold, it won't help many applications.

I plan to add a new view with graphs that will show exactly what the problem
is.

Cheers!
Miquel


PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see
what improvements there have been since then
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20100325/56cd8869/attachment.html>