[Cython] speed.pypy.org

[Stefan Behnel]

Stefan Behnel, 11.04.2011 15:08:
> I'm currently discussing with Maciej Fijalkowski (PyPy) how to get Cython
> running on speed.pypy.org (that's what I wrote "cythonrun" for). If it
> works out well, we may have it up in a couple of days.

... or maybe not. It may take a little longer due to lack of time on his side.


> I would expect that Cython won't be a big winner in this game, given that
> it will only compile plain untyped Python code. It's also going to fail
> entirely in some of the benchmarks. But I think it's worth having it up
> there, simply as a way for us to see where we are performance-wise and to
> get quick (nightly) feed-back about optimisations we try. The benchmark
> suite is also a nice set of real-world Python code that will allow us to
> find compliance issues.

Ok, here's what I have so far. I fixed a couple of bugs in Cython and got 
at least some of the benchmarks running. Note that they are actually simple 
ones, only a single module. Basically all complex benchmarks fail due to 
known bugs, such as Cython def functions not accepting attribute 
assignments (e.g. on wrapping). There's also a problem with code that uses 
platform specific names conditionally, such as WindowsError when running on 
Windows. Cython complains about non-builtin names here. I'm considering to 
turn that into a visible warning instead of an error, so that the name 
would instead be looked up dynamically to let the code fail at runtime 
*iff* it reaches the name lookup.

Anyway, here are the numbers. I got them with "auto_cpdef" enabled, 
although that doesn't even seem to make that a big difference. The baseline 
is a self-compiled Python 2.7.1+ (about a month old).

Stefan


### ai ###
Min: 0.402407 -> 0.362190: 1.1110x faster
Avg: 0.408784 -> 0.366898: 1.1142x faster
Significant (t=10.017195, a=0.95)
Stddev: 0.00824 -> 0.00442: 1.8668x smaller


### chaos ###  (with a bug fixed in the benchmark itself)
Min: 0.393362 -> 0.231932: 1.6960x faster
Avg: 0.401941 -> 0.234089: 1.7170x faster
Significant (t=36.128709, a=0.95)
Stddev: 0.01004 -> 0.00267: 3.7538x smaller


### crypto_pyaes ###
Min: 2.629560 -> 1.276433: 2.0601x faster
Avg: 2.639409 -> 1.277742: 2.0657x faster
Significant (t=368.652396, a=0.95)
Stddev: 0.00812 -> 0.00153: 5.3215x smaller


### fannkuch ###
Min: 1.512630 -> 0.853309: 1.7727x faster
Avg: 1.522860 -> 0.860237: 1.7703x faster
Significant (t=118.573908, a=0.95)
Stddev: 0.00880 -> 0.00887: 1.0073x larger


### float ###
Min: 0.452620 -> 0.343341: 1.3183x faster
Avg: 0.475137 -> 0.349356: 1.3600x faster
Significant (t=9.575876, a=0.95)
Stddev: 0.02838 -> 0.00757: 3.7489x smaller


### go ###
Min: 0.758998 -> 0.491929: 1.5429x faster
Avg: 0.764110 -> 0.496518: 1.5389x faster
Significant (t=90.848407, a=0.95)
Stddev: 0.00400 -> 0.00523: 1.3096x larger


### nbody_modified ###
Min: 0.399168 -> 0.197931: 2.0167x faster
Avg: 0.401379 -> 0.203112: 1.9762x faster
Significant (t=42.377829, a=0.95)
Stddev: 0.00293 -> 0.01004: 3.4337x larger


### raytracesimple ###   (module renamed from "raytrace-simple")
Min: 2.016425 -> 1.182970: 1.7045x faster
Avg: 2.030030 -> 1.192164: 1.7028x faster
Significant (t=78.219481, a=0.95)
Stddev: 0.02184 -> 0.00983: 2.2211x smaller


### richards ###
Min: 0.286723 -> 0.162430: 1.7652x faster
Avg: 0.289933 -> 0.165193: 1.7551x faster
Significant (t=52.898468, a=0.95)
Stddev: 0.00392 -> 0.00352: 1.1127x smaller