[pypy-dev] Unicode encode/decode speed

[Maciej Fijalkowski]

On Tue, Feb 12, 2013 at 8:14 PM, Eleytherios Stamatogiannakis
<estama at gmail.com> wrote:
> On 12/02/13 11:04, Maciej Fijalkowski wrote:
>>
>>
>> I would like to see some evidence about it. Did you try valgrind?
>>
>> Cheers,
>> fijal
>>
>
> Even better, we wanted to find a way for you to be able to test it by
> yourselves, so we tried to create a representative synthetic benchmark.
>
> Surprisingly when we retested the benchmark that we had previously posted
> here in this mailing list, we found that the performance profile is very
> similar to the one slow query that i've talked about in my recent emails.
>
> To make it easier i'll repeat the freshened instructions (from the old
> email) of how to run that benchmark. Also attached is the updated (and
> heavily optimized) MSPW:
>
> --repost--
>
> To run it you'll need latest madIS. You can clone it using:
>
> hg clone https://code.google.com/p/madis/
>
> For running the test with CPython you'll need:
>
> CPython 2.7 + APSW:
>
> https://code.google.com/p/apsw/
>
> For PyPy you'll need MPSW renamed to "apsw.py" (the attached MPSW is already
> renamed to "apsw.py").
> Move "apsw.py" to pypy's "site-packages" directory. For MSPW to work in
> PyPy, you'll also need CFFI and "libsqlite3" installed.
>
> To run the test with PyPy:
>
> pypy mterm.py < mspw_bench.sql
>
> or with CPython
>
> python mterm.py < mspw_bench.sql
>
> The timings of "mspw_bench" that we get are:
>
> CPython 2.7 + APSW: ~ 2.6sec
> PyPy + MSPW: ~ 4sec
>
> There are two ways to adjust the processing load of mspw_bench.
>
> One is to change the value in "range(xxxxx)". This will in essence create a
> bigger/smaller "synthetic text". This puts more pressure on CPython's/pypy's
> side.
>
> The other way is to adjust the window size of textwindow(t, xx, xx). This
> puts more pressure on the wrapper (APSW/MSPW)  because it changes the number
> of columns that CPython/PyPy have to send to SQLite (they are send one value
> at a time).
>
> --/repost--
>
> Attached you'll find our latest MSPW (renamed to "apsw.py") and
> "mspw_bench.sql"
>
> Also we are looking into adding a special ffi.string_decode_UTF8 in CFFI's
> backend to reduce the number of calls that are needed to go from utf8_char*
> to PyPy's unicode.
>
> Do you thing that such an addition would be worthwhile?
>
> Thank you,
>
> lefteris
>
>
>
>
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
>

Hey

I have serious trouble running apsw. Message I got so far:

/home/fijal/.virtualenvs/cffi/local/lib/python2.7/site-packages/apsw.so:
undefined symbol: sqlite3_uri_boolean