[pypy-dev] benchmarking input

Stefan Behnel stefan_ml at behnel.de
Fri Oct 2 22:20:23 CEST 2009


Anders Hammarquist wrote:
> Anyway, what I was thinking about, and need input on, is how to get
> at the interpreters to run the benchmark.
[...]

I wonder, would it be possible to add Cython to the benchmark loop? I would
love to see it compared to PyPy, simply because both projects aim to
compile Python code to C code (amongst other things, obviously).

I know that Cython can't currently compete with PyPy in terms of feature
completeness - it clearly lacks some very important features of the Python
language, so it won't be able to run all benchmarks for a while, and the
comparison would easily show where the black spots are that need fixing.

Just out of curiosity (and to wet your appetite :), I ran PyPy's richards
benchmark unmodified in Cython (latest cython-unstable) and got this:

  python2.6 -c 'import richards; richards.main()'
  Richards benchmark (Python) starting... [...]
  finished.
  Total time for 10 iterations: 3.98 secs
  Average time per iteration: 398.44 ms

compared to CPython 2.6.2:

  python2.6 -c 'import richards; richards.main()'
  Richards benchmark (Python) starting... [...]
  finished.
  Total time for 10 iterations: 4.86 secs
  Average time per iteration: 485.97 ms

That's almost 20% faster IMO and not bad at all, given that Cython's main
performance feature (C typing) wasn't used. When I use an external .pxd
file (attached) to redeclare the classes as extension types and to add a C
nature to their methods (still without any benchmark code modifications), I
get this:

  python2.6 -c 'import richards; richards.main(iterations=10)'
  Richards benchmark (Python) starting... [...]
  finished.
  Total time for 10 iterations: 0.99 secs
  Average time per iteration: 99.28 ms

That's almost a factor of five compared to CPython.

If possible, I would like to add both a normal Cython compiler run and a
pxd enabled run to the benchmark comparison with PyPy and CPython. Any
chance this could be integrated?

I'm asking now, because I imagine that the benchmarking framework will have
to integrate the Cython compiler somehow, maybe using distutils or
on-the-fly compilation with pyximport.

Stefan
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: richards.pxd
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20091002/d732fa52/attachment.ksh>


More information about the Pypy-dev mailing list