Re: [Speed] Cython's view on a common benchmark suite
Maciej Fijalkowski, 02.02.2012 14:35:
Oh, we have that feature, it's called CPython. The thing is that Cython doesn't get to see the generated sources, so it won't compile them and instead, CPython ends up executing the code at normal interpreted speed. So there's nothing gained by running the benchmark at all. And even if we found a way to hook into this machinery, I doubt that the static compiler overhead would make this any useful. The whole purpose of generating code is that it likely will not look the same the next time you do it (well, outside of benchmarks, that is), so even a cache is unlikely to help much for real code. It's like PyPy running code in interpreting mode before it gets compiled, except that Cython will never compile this code, even if it turns out to be worth it.
Personally, I rather consider it a feature that users can employ exec() from their Cython code to run code in plain CPython (for whatever reason).
Yes, ok, but I believe this should mean "Cython does not give speedups on this benchmark" and not "we should modify the benchmark".
Oh, I hadn't suggested to modify it. I was merely stating (as part of a longer list) that it's of no use specifically to Cython. I.e., if there's something to gain from having the benchmark runs take less time by disabling benchmarks for specific runtimes, it's one of the candidates on our side.
Stefan
On Thu, Feb 2, 2012 at 3:40 PM, Stefan Behnel <stefan_ml@behnel.de> wrote:
Maciej Fijalkowski, 02.02.2012 14:35:
Oh, we have that feature, it's called CPython. The thing is that Cython doesn't get to see the generated sources, so it won't compile them and instead, CPython ends up executing the code at normal interpreted speed. So there's nothing gained by running the benchmark at all. And even if we found a way to hook into this machinery, I doubt that the static compiler overhead would make this any useful. The whole purpose of generating code is that it likely will not look the same the next time you do it (well, outside of benchmarks, that is), so even a cache is unlikely to help much for real code. It's like PyPy running code in interpreting mode before it gets compiled, except that Cython will never compile this code, even if it turns out to be worth it.
Personally, I rather consider it a feature that users can employ exec() from their Cython code to run code in plain CPython (for whatever reason).
Yes, ok, but I believe this should mean "Cython does not give speedups on this benchmark" and not "we should modify the benchmark".
Oh, I hadn't suggested to modify it. I was merely stating (as part of a longer list) that it's of no use specifically to Cython. I.e., if there's something to gain from having the benchmark runs take less time by disabling benchmarks for specific runtimes, it's one of the candidates on our side.
Stefan
Oh ok, I misread you then, sorry.
I think having a dedicated speed.python machine is specifically so that we can run benchmarks however much we want :) At least Cython does not compile for like an hour...
participants (2)
-
Maciej Fijalkowski
-
Stefan Behnel