[pypy-dev] could it be that cpyext is faster than python + cffi?

[Armin Rigo]

Hi Matti,

On 13 February 2015 at 14:46, Matti Picus <matti.picus at gmail.com> wrote:
> I am close to releasing a blog post about numpy.linalg

A comment: saying "In order to test it out, download PyPy 2.5.0 or
later, and install the pypy version of numpy" in the conclusion is, in
my opinion, both too late and not precise enough.  Please put in the
introduction two lines that say clearly something like:

    In order to install the version of numpy this blog post is talking
about, run::

         pypy -m pip install git+https://bitbucket.org/pypy/numpy.git

> $pypy inverse.py 10
> $USE_CPYEXT=yes pypy inverse.py 10
> Any thoughts?

That's probably because the cffi version does more copying of the
data, and more slowly.  It's not really about cffi versus cpyext.
Here you are comparing a version that, with cpyext, calls just "inv()"
once (with the overhead of cpyext once), with a version that uses cffi
to do in Python a whole layer of manipulations that was done in C (the
@TYPE at _inv() function in _umath_linalg.c.src and its callees).  I
think it is not surprizing that the former is faster.

If you want to find a design that makes sense performance-wise, I
would try to look for one where this C layer of manipulations is still
C code.  You would still having a family of functions called
"@TYPE at _inv()" that are called only once --- you can select the
correct "@TYPE at _inv()" to call from Python code.

In other words, working C code shouldn't be rewritten in Python.  Only
the parts of C code with strong dependencies on the CPython C API
needs to be.

Note that I just had a quick look, I may be missing some issues.


A bientôt,

Armin.