[pypy-dev] CFFI speed results

[Eleytherios Stamatogiannakis]

Hello,

We have been testing CFFI here for the purpose of speeding up madIS [*], 
and here are some preliminary results.

First of all, under pypy, CFFI is a *lot* faster than ctypes. In 
callback microbenchmarks (using quicksort to call the callbacks), 
pypy/CFFI had ~8-10 times less overhead than pypy/ctypes.

Using libsqlite3 as the C library calling the callbacks, we found that, 
compared to Python/APSW [**] the callback overhead of pypy/CFFI was 3-4 
times larger.

To be able to run madIS, we started from pypy's _sqlite3.py (which uses 
ctypes) and did a rough conversion of it to CFFI. We then changed it to 
be APSW compatible. In the rest of the email, we'll refer to this APSW 
compatible CFFI wrapper, as MSPW.

The end results were very encouraging. Using madIS and for SQL queries 
that didn't produce a lot of columns, we could get speedups up to 2 times.

For SQL queries that produce a lot of columns, we get slowdowns of 2 - 
2.5 times. Running a profiler on pypy/madIS, we found out that the main 
bottleneck is not callback performance any more, but it is regular 
pypy->C call speed.

Whereas Python/APSW spends most of its time on madIS' Python execution 
of functions. Pypy/MSPW spends most of its time (~45-50%) on calling a 
libsqlite3 function that passes a Python string back to libsqlite3, and 
10% in pypy's string.encode('utf-8') function.

So for pypy/MSPW most of the time (55-60%) is spend just to pass a 
string back to libsqlite3.

In Python/APSW the time spended to pass the string back to libsqlite3 is 
<1%.

The libsqlite3 function's header info is:

void sqlite3_result_text(sqlite3_context*, const char*, int, 
void(*)(void*));

In the main query that we've used in our tests, above libsqlite3 
function is called 1.1 Million times.

The times of this query, running under the different options are:

  Python/APSW: 40s
  pypy/MSPW: 2m 3s
  pypy/APSW: 2m 21s

Best regards,

l.

[*] https://code.google.com/p/madis/
[*] https://code.google.com/p/apsw/