[Python-Dev] VM and Language summit info for those not at Pycon (and those that are!)

[Stefan Behnel]

Jesse Noller, 20.03.2011 13:51:
> ...snip
>
>> IMHO, taking modules that currently only have a C implementation due to
>> performance constraints and rewriting them in Cython is a much more
>> worthwhile thing to do than adding an alternative pure Python implementation
>> that other Python runtimes wouldn't use anyway. And at least IronPython
>> could soon benefit directly from a Cython implementation as well.
>>
>> Stefan
>
> The other python implementation expressed serious interest, and are
> willing to help support a shared standard library, and shared modules.
> So saying they "won't use them anyway" may apply to things such as the
> io module, but is far from the truth for the entire standard library.

I guess someone would have to look through the stdlib and make a list of 
suitable candidates for Cython compilation and/or Python/Cython/C 
reimplementations at this point.


> You're also asking that we depend on Cython within core

It's a substantial dependency, sure. The question is: what's more work, 
having to install a specific version of Cython in order to work on CPython, 
or having to get fluent in C + C-API first?

I like the way the Jython people put it, slightly adapted into "We write C 
so you don't have to".


> which while
> it has it's benefits, I think warrants a PEP and a working
> implementation to show the potential speedups you're talking about
> before we can agree to include it/depend on it.

We will have a Cython core developers workshop next weekend. Maybe we can 
find a bit of time there to run the then-merged-together-bleeding-edge 
Cython over the standard library and see what that gives.

It's not easy to find good benchmarks, though. Most of what the 
PyPy/Unladen developers settled on so far isn't very interesting for the 
way Cython works. It's usually a bit of work to make the code substantially 
faster by providing enough type annotations to let the compiler drop 
critical Python code into C. We did that for a couple of benchmarks, but 
lost interest as there wasn't much to win - all we could show was that, by 
changing the code enough to make it violate the usual constraints of a fair 
benchmark comparison, you could make it run as fast as C code without 
writing C code. Not much news to us and nothing that would integrate in any 
acceptable way into speed.pypy.org.

If anyone knows about a good benchmark for a currently pure Python standard 
library module, preferably a smaller, self-contained one that's somewhat 
computationally intensive, I'd be happy to hear about it.

Stefan