[pypy-dev] Unicode encode/decode speed (cont)

[Eleytherios Stamatogiannakis]

On 18/02/13 18:44, Maciej Fijalkowski wrote:
> On Mon, Feb 18, 2013 at 6:20 PM, Eleytherios Stamatogiannakis
> <estama at gmail.com> wrote:
>> We have found another (very simple) madIS query where PyPy is around 250x
>> slower that CPython:
>>
>> CPython: 314msec
>> PyPy: 1min 16sec
>>
>> The query if you would like to test it yourself is the following:
>>
>> select  count(*)  from   (file  'some_big_text_file.txt' limit 100000);
>>
>> To run it you'll need some big text file containing at least 100000 text
>> lines (we have run above query with a very big XML file). You can also run
>> above query with a lower limit (the behaviour will be the same) as such:
>>
>> select  count(*)  from   (file  'some_big_text_file.txt' limit 10000);
>>
>> Be careful for the file to not have a csv, tsv, json, db or gz ending
>> because a different code path inside the "file" operator will be taken than
>> the one for simple text files.
>>
>> l.
>>
>>
>> _______________________________________________
>> pypy-dev mailing list
>> pypy-dev at python.org
>> http://mail.python.org/mailman/listinfo/pypy-dev
>
> Hey
>
> I would be incredibly convinient if you can change it to be a
> standalone benchmark (say reading large string from a file and
> decoding it in a whole or in pieces);
>

As it involves SQLite, CFFI and Python, it is very hard to extract the 
full execution path that madIS goes through even in a simple query like 
this.

Nevertheless we extracted a part of the pure Python execution path, and 
PyPy is around 50% slower than CPython:

CPython: 21 sec
PyPy: 33 sec

The full madIS execution path involves additional CFFI calls and 
callbacks (from SQLite) to pass the data to SQLite.

To run the test.py:

test.py big_text_file

l.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: text/x-python
Size: 463 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20130218/54d0dd10/attachment.py>