[Speed] Cython's view on a common benchmark suite

[Stefan Behnel]

Maciej Fijalkowski, 02.02.2012 14:35:
>> Oh, we have that feature, it's called CPython. The thing is that Cython
>> doesn't get to see the generated sources, so it won't compile them and
>> instead, CPython ends up executing the code at normal interpreted speed. So
>> there's nothing gained by running the benchmark at all. And even if we
>> found a way to hook into this machinery, I doubt that the static compiler
>> overhead would make this any useful. The whole purpose of generating code
>> is that it likely will not look the same the next time you do it (well,
>> outside of benchmarks, that is), so even a cache is unlikely to help much
>> for real code. It's like PyPy running code in interpreting mode before it
>> gets compiled, except that Cython will never compile this code, even if it
>> turns out to be worth it.
>>
>> Personally, I rather consider it a feature that users can employ exec()
>> from their Cython code to run code in plain CPython (for whatever reason).
> 
> Yes, ok, but I believe this should mean "Cython does not give speedups
> on this benchmark" and not "we should modify the benchmark".

Oh, I hadn't suggested to modify it. I was merely stating (as part of a
longer list) that it's of no use specifically to Cython. I.e., if there's
something to gain from having the benchmark runs take less time by
disabling benchmarks for specific runtimes, it's one of the candidates on
our side.

Stefan