Hi all, thanks for the responses, but I guess I should have been more explicit -- I'm curious about *why* PyPy is slow on existing extension modules and why people are being steered away from them.  I completely support the push to move away from CPython extension modules, but I'm not sure it's reasonable to expect that programmers will rewrite all the extension modules they use.

Put another way, I understand how having a JIT-understandable cffi module will be faster on PyPy than an extension module, but what I don't quite understand is why CPython extension modules have to be slower on PyPy than they are on CPython.  I'm not saying that extension modules should be sped up by PyPy, but I'm curious why they have a reputation for being slower.


On Wed, Mar 26, 2014 at 1:31 AM, Yury V. Zaytsev <yury@shurup.com> wrote:
On Tue, 2014-03-25 at 16:19 -0700, Kevin Modzelewski wrote:
>
> I'm curious, since I've heard a number of people mention that
> extension modules are the primary reason that PyPy is slower than
> cPython for their code; definitely an improvement over "PyPy doesn't
> run my code at all", but it's made me curious about whether or not /
> why it has to be that way.

In my opinion, it all depends on how do you use CPyExt and what your
extension modules are for. There are two scenarios here (or combinations
thereof) that I think cover most of the use cases:

1) You use C extensions to make it faster.
2) You use C extensions to steer external processes.

Ideally with PyPy you should be able to drop (1) altogether and write
nice Python code that JIT will be able to optimize sometimes even better
than hand-written C code, so here the answer would be "don't use
extensions".

Now, if as a part of (2) you are doing some lengthy processing entirely
outside PyPy, this might still just as fast as with CPython with CPyExt,
but if the calls to your foreign functions are short and/or you are
transferring a lot of data C <-> PyPy, then there you go...

Personally, I've been using CPyExt and I'm very happy about it, because
the function calls take a long time, and whatever happens outside
doesn't have much to do with objects in PyPy land.

However, if my requirements were different, I would have rather
re-written everything using cffi, from what I understood it can deliver
comparable performance to cPython, and also it works both for PyPy and
cPython, not just PyPy...

--
Sincerely yours,
Yury V. Zaytsev