Hi all, On Wed, Feb 6, 2013 at 12:26 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:

it's probably already fixed on jitframe-on-heap which we aim to merge

Just answering this mail for the records: yes, on windows these tests pass on jitframe-on-heap. Armin

Hi Fijal, On Mon, Feb 11, 2013 at 9:59 AM, Maciej Fijalkowski <fijall@gmail.com> wrote:

How so? The 32bit support is by far not done

Dunno? These two tests (from "test_basic.py -k test_float") also pass when running on linux32 fwiw. A bientôt, Armin.

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com>

Right now we are using PyPy's "codecs.utf_8_encode" and "codecs.utf_8_decode" to do this conversion.

It's the most direct way to use the utf-8 conversion functions.

It there a faster way to do these conversions (encoding, decoding) in PyPy? Does CPython do something more clever than PyPY, like storing unicodes with full ASCII char content, in an ASCII representation?

Over years, utf-8 conversions have been heavily optimized in CPython: allocate short buffers on the stack, use aligned reads, quick check for ascii-only content (data & 0x80808080)... All things that pypy does not. But I tried some "timeit" runs, and pypy is often faster that CPython, and never much slower. Do your strings have many non-ascii characters? what's the len(utf8)/len(unicode) ratio? -- Amaury Forgeot d'Arc

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com>

On 11/02/13 18:13, Amaury Forgeot d'Arc wrote:

...

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com <mailto:estama@gmail.com>>

Right now we are using PyPy's "codecs.utf_8_encode" and "codecs.utf_8_decode" to do this conversion.

It's the most direct way to use the utf-8 conversion functions.

It there a faster way to do these conversions (encoding, decoding) in PyPy? Does CPython do something more clever than PyPY, like storing unicodes with full ASCII char content, in an ASCII representation?

Over years, utf-8 conversions have been heavily optimized in CPython: allocate short buffers on the stack, use aligned reads, quick check for ascii-only content (data & 0x80808080)... All things that pypy does not.

But I tried some "timeit" runs, and pypy is often faster that CPython, and never much slower.

This is odd. Maybe APSW uses some other CPython conversion API? Because the conversion overhead is not visible on CPython + APSW profiles.

Which kind of profiler are you using? It possible that CPython builtin functions are not profiled the same way as PyPy's.

Do your strings have many non-ascii characters?

...

what's the len(utf8)/len(unicode) ratio?

Our current tests, are using plain ASCII input (imported into sqlite3) which:

- Go from sqlite3 (UTF-8) -> PyPy (unicode) - PyPy (unicode) -> sqlite3 (UTF-8).

So i guess the len(utf-8)/len(unicode) = 1/4 (assuming 1 byte per char for ASCII (UTF-8) and 4 bytes per char for PyPy's unicode storage)

No, my question was about the number of non-ascii characters: s = u"SomeUnicodeString" 1.0 * len(s.encode('utf8')) / len(s) PyPy allocates the StringBuffer upfront, and must realloc to cope with multibytes characters. For English text, ratio is 1.0; for Greek, it will be close to 2.0. -- Amaury Forgeot d'Arc

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com>

...

Which kind of profiler are you using? It possible that CPython builtin functions are not profiled the same way as PyPy's.

lsprofcalltree.py .

From APSW's source code, i think that it uses this API:

(in cursor.c) PyUnicode_DecodeUTF8

Maybe lsprofcalltree doesn't profile it?

Indeed. CPU cost is hidden in the cursor method. -- Amaury Forgeot d'Arc

I've also heard great things about `perf` if you're on Linux. Alex On Mon, Feb 11, 2013 at 12:03 PM, Maciej Fijalkowski <fijall@gmail.com>wrote:

On Mon, Feb 11, 2013 at 7:39 PM, Amaury Forgeot d'Arc <amauryfa@gmail.com> wrote:

...

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com>

...

...

Which kind of profiler are you using? It possible that CPython builtin functions are not profiled the same way as PyPy's.

lsprofcalltree.py .

From APSW's source code, i think that it uses this API:

(in cursor.c) PyUnicode_DecodeUTF8

Maybe lsprofcalltree doesn't profile it?

Indeed. CPU cost is hidden in the cursor method.

-- Amaury Forgeot d'Arc

_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

I would suggest using valgrind. It's a very good (albeit very slow) tool for seeing C-level performance. I remember seeing it both for CPython and PyPy when trying. Can you try yourself? _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

2013/2/12 Elefterios Stamatogiannakis <estama@gmail.com>

On 11/2/2013 7:39 μμ, Amaury Forgeot d'Arc wrote:

...

2013/2/11 Eleytherios Stamatogiannakis <estama@gmail.com <mailto:estama@gmail.com>>

> > Which kind of profiler are you using? It possible that CPython builtin > functions are not profiled the same way as PyPy's.

lsprofcalltree.py .

From APSW's source code, i think that it uses this API:

(in cursor.c) PyUnicode_DecodeUTF8

Maybe lsprofcalltree doesn't profile it?

Indeed. CPU cost is hidden in the cursor method.

Thanks Amaury for looking into this,

Assuming that PyPy's "codecs.utf_8_decode" is slower when used with CFFI than using PyUnicode_DecodeUTF8 in CPython.

Is there anything that can be done in CFFI that would have the same performance as PyUnicode_DecodeUTF8 (and the same for encode)

First, codecs.utf_8_decode has nothing to do with CFFI... Then, do we have evidence that the utf8 codec is enough to explain the different performance? Since your data is only ASCII, it would be interesting to use the ASCII encoding: try to replace PyUnicode_DecodeUTF8 by PyUnicode_DecodeASCII and codecs.utf_8_decode by codecs.ascii_decode -- Amaury Forgeot d'Arc

On 12/02/13 11:04, Maciej Fijalkowski wrote:

I would like to see some evidence about it. Did you try valgrind?

Cheers, fijal

Even better, we wanted to find a way for you to be able to test it by yourselves, so we tried to create a representative synthetic benchmark. Surprisingly when we retested the benchmark that we had previously posted here in this mailing list, we found that the performance profile is very similar to the one slow query that i've talked about in my recent emails. To make it easier i'll repeat the freshened instructions (from the old email) of how to run that benchmark. Also attached is the updated (and heavily optimized) MSPW: --repost-- To run it you'll need latest madIS. You can clone it using: hg clone https://code.google.com/p/madis/ For running the test with CPython you'll need: CPython 2.7 + APSW: https://code.google.com/p/apsw/ For PyPy you'll need MPSW renamed to "apsw.py" (the attached MPSW is already renamed to "apsw.py"). Move "apsw.py" to pypy's "site-packages" directory. For MSPW to work in PyPy, you'll also need CFFI and "libsqlite3" installed. To run the test with PyPy: pypy mterm.py < mspw_bench.sql or with CPython python mterm.py < mspw_bench.sql The timings of "mspw_bench" that we get are: CPython 2.7 + APSW: ~ 2.6sec PyPy + MSPW: ~ 4sec There are two ways to adjust the processing load of mspw_bench. One is to change the value in "range(xxxxx)". This will in essence create a bigger/smaller "synthetic text". This puts more pressure on CPython's/pypy's side. The other way is to adjust the window size of textwindow(t, xx, xx). This puts more pressure on the wrapper (APSW/MSPW) because it changes the number of columns that CPython/PyPy have to send to SQLite (they are send one value at a time). --/repost-- Attached you'll find our latest MSPW (renamed to "apsw.py") and "mspw_bench.sql" Also we are looking into adding a special ffi.string_decode_UTF8 in CFFI's backend to reduce the number of calls that are needed to go from utf8_char* to PyPy's unicode. Do you thing that such an addition would be worthwhile? Thank you, lefteris

On 13/02/13 10:08, Maciej Fijalkowski wrote:

Hey

I have serious trouble running apsw. Message I got so far:

/home/fijal/.virtualenvs/cffi/local/lib/python2.7/site-packages/apsw.so: undefined symbol: sqlite3_uri_boolean

Thanks Maciej for looking into it, Which version of APSW have you tried to install and how? Debian/Ubuntu based systems, include APSW in package: python-apsw You can also try to install APSW yourself. Instructions: http://apidoc.apsw.googlecode.com/hg/build.html#recommended and (some more details from madIS' docs) http://doc.madis.googlecode.com/hg/install.html l.

Hi, On Tue, Feb 12, 2013 at 7:14 PM, Eleytherios Stamatogiannakis <estama@gmail.com> wrote:

Also we are looking into adding a special ffi.string_decode_UTF8 in CFFI's backend to reduce the number of calls that are needed to go from utf8_char* to PyPy's unicode.

A first note: I'm wondering why you need to convert from utf-8-that-contains-only-ascii, to unicode, and back. What is the point of having unicode strings in the first place? Can't you just pass around your complete program plain non-unicode strings? If not, then indeed, it would make (a bit of) sense to have ways to convert directly between "char *" and unicode strings, in both directions, assuming utf-8. This could be done with an API like: ffi.encode_utf8(unicode_string) -> new_char*_cdata ffi.encode_utf8(unicode_string, target_char*_cdata, maximum_length) ffi.decode_utf8(char*_cdata, [length]) -> unicode_string Alternatively, we could accept unicode strings whenever a "char*" is expected and encode it to utf-8, but that sounds a bit too magical. A bientôt, Armin.

On 17/02/13 11:43, Armin Rigo wrote:

Hi,

On Tue, Feb 12, 2013 at 7:14 PM, Eleytherios Stamatogiannakis <estama@gmail.com> wrote:

...

Also we are looking into adding a special ffi.string_decode_UTF8 in CFFI's backend to reduce the number of calls that are needed to go from utf8_char* to PyPy's unicode.

A first note: I'm wondering why you need to convert from utf-8-that-contains-only-ascii, to unicode, and back. What is the point of having unicode strings in the first place? Can't you just pass around your complete program plain non-unicode strings?

The problem is that SQlite internally uses UTF-8. So you cannot know in advance if the char* that you get from it is plain ASCII or a UTF-8 encoded Unicode. So we end up always converting to Unicode from the char* that SQlite returns. When sending to it, we have different code paths for Python's str() and unicode() string representations. Unfortunately, due to the nature of our data (its multilingual), and to make our life easier when we code our relational operators (written in Python), we always convert to Unicode inside our operators. So the str() path inside the MSPW SQLite wrapper, mostly sits unused.

If not, then indeed, it would make (a bit of) sense to have ways to convert directly between "char *" and unicode strings, in both directions, assuming utf-8. This could be done with an API like:

ffi.encode_utf8(unicode_string) -> new_char*_cdata ffi.encode_utf8(unicode_string, target_char*_cdata, maximum_length) ffi.decode_utf8(char*_cdata, [length]) -> unicode_string

Alternatively, we could accept unicode strings whenever a "char*" is expected and encode it to utf-8, but that sounds a bit too magical.

An API like the one you propose would be very nice, and IMHO would give a substantial speedup. May i suggest, that for generality purposes, the same API functions should also be added for UTF-16, UTF-32 ? Thanks Armin and Maciej for looking into this, l.

We have found another (very simple) madIS query where PyPy is around 250x slower that CPython: CPython: 314msec PyPy: 1min 16sec The query if you would like to test it yourself is the following: select count(*) from (file 'some_big_text_file.txt' limit 100000); To run it you'll need some big text file containing at least 100000 text lines (we have run above query with a very big XML file). You can also run above query with a lower limit (the behaviour will be the same) as such: select count(*) from (file 'some_big_text_file.txt' limit 10000); Be careful for the file to not have a csv, tsv, json, db or gz ending because a different code path inside the "file" operator will be taken than the one for simple text files. l.

On 18/02/13 18:44, Maciej Fijalkowski wrote:

On Mon, Feb 18, 2013 at 6:20 PM, Eleytherios Stamatogiannakis <estama@gmail.com> wrote:

...

We have found another (very simple) madIS query where PyPy is around 250x slower that CPython:

CPython: 314msec PyPy: 1min 16sec

The query if you would like to test it yourself is the following:

select count(*) from (file 'some_big_text_file.txt' limit 100000);

To run it you'll need some big text file containing at least 100000 text lines (we have run above query with a very big XML file). You can also run above query with a lower limit (the behaviour will be the same) as such:

select count(*) from (file 'some_big_text_file.txt' limit 10000);

Be careful for the file to not have a csv, tsv, json, db or gz ending because a different code path inside the "file" operator will be taken than the one for simple text files.

l.

_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

Hey

I would be incredibly convinient if you can change it to be a standalone benchmark (say reading large string from a file and decoding it in a whole or in pieces);

As it involves SQLite, CFFI and Python, it is very hard to extract the full execution path that madIS goes through even in a simple query like this. Nevertheless we extracted a part of the pure Python execution path, and PyPy is around 50% slower than CPython: CPython: 21 sec PyPy: 33 sec The full madIS execution path involves additional CFFI calls and callbacks (from SQLite) to pass the data to SQLite. To run the test.py: test.py big_text_file l.

So, iter(file).next() is slow? Alex On Mon, Feb 18, 2013 at 10:51 AM, Amaury Forgeot d'Arc <amauryfa@gmail.com>wrote:

2013/2/18 Eleytherios Stamatogiannakis <estama@gmail.com>

...

On 18/02/13 18:44, Maciej Fijalkowski wrote:

...

On Mon, Feb 18, 2013 at 6:20 PM, Eleytherios Stamatogiannakis <estama@gmail.com> wrote:

...

We have found another (very simple) madIS query where PyPy is around 250x slower that CPython:

CPython: 314msec PyPy: 1min 16sec

The query if you would like to test it yourself is the following:

select count(*) from (file 'some_big_text_file.txt' limit 100000);

To run it you'll need some big text file containing at least 100000 text lines (we have run above query with a very big XML file). You can also run above query with a lower limit (the behaviour will be the same) as such:

select count(*) from (file 'some_big_text_file.txt' limit 10000);

Be careful for the file to not have a csv, tsv, json, db or gz ending because a different code path inside the "file" operator will be taken than the one for simple text files.

l.

______________________________**_________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/**mailman/listinfo/pypy-dev<http://mail.python.org/mailman/listinfo/pypy-dev>

Hey

I would be incredibly convinient if you can change it to be a standalone benchmark (say reading large string from a file and decoding it in a whole or in pieces);

As it involves SQLite, CFFI and Python, it is very hard to extract the full execution path that madIS goes through even in a simple query like this.

Nevertheless we extracted a part of the pure Python execution path, and PyPy is around 50% slower than CPython:

CPython: 21 sec PyPy: 33 sec

The full madIS execution path involves additional CFFI calls and callbacks (from SQLite) to pass the data to SQLite.

To run the test.py:

test.py big_text_file

Most of the time is spent in file iteration. I added f = f.read().splitlines() and the query is almost instant.

-- Amaury Forgeot d'Arc

_______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev

-- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

On 18/02/13 21:15, Amaury Forgeot d'Arc wrote:

2013/2/18 Alex Gaynor <alex.gaynor@gmail.com <mailto:alex.gaynor@gmail.com>>

So, iter(file).next() is slow?

Yes, but only with "rU" mode. My benchmark with yesterday's build:

$ ~/pypy/pypy-c-jit-60005-0f1e91da6cb2-linux64/bin/pypy -m timeit "fp = open('/tmp/large-text-file'); list(fp)" 10 loops, best of 3: 43.5 msec per loop $ ~/pypy/pypy-c-jit-60005-0f1e91da6cb2-linux64/bin/pypy -m timeit "fp = open('/tmp/large-text-file', 'rU'); list(fp)" 10 loops, best of 3: 638 msec per loop

15 times slower...

Yes you are right. We rerun the query without the 'rU' and the result is: CPython: 328 msec PyPy: 443 msec PyPy (with 'rU'): 1 min 17 sec So the main culprit of PyPy's slowdown is 'rU' option in open. Thanks for looking into it. l.

2013/2/19 Maciej Fijalkowski <fijall@gmail.com>

On Tue, Feb 19, 2013 at 2:09 PM, Eleytherios Stamatogiannakis <estama@gmail.com> wrote:

...

So the main culprit of PyPy's slowdown is 'rU' option in open.

Thanks for looking into it.

Is this yet-another-fault-of-streamio?

Not quite. Even a implementation based on fread() would need to care of these universal newlines and tune the usage of the various buffers. BTW, I tried io.open, and surprisingly the "rb" mode is twice slower as "rU". I guess that's because our io.Buffered is missing a dedicated "readline" method. -- Amaury Forgeot d'Arc

On 18/02/13 20:21, Amaury Forgeot d'Arc wrote:

2013/2/18 Eleytherios Stamatogiannakis <estama@gmail.com <mailto:estama@gmail.com>>

We have found another (very simple) madIS query where PyPy is around 250x slower that CPython:

CPython: 314msec PyPy: 1min 16sec

The query if you would like to test it yourself is the following:

select count(*) from (file 'some_big_text_file.txt' limit 100000);

Are you really running with mpsw.py? For me, the C (=cpyext based) version of apsw works (slowly), but mpsw gives me:

From callback <function xOpen at 0x0000000005575ad8>: Traceback (most recent call last): File "/home/amauryfa/python/madis/apsw.py", line 924, in xOpen self._vtcursorcolumn.append(instance.Column) AttributeError: Cursor instance has no attribute 'Column'

Most probably you are using the ZIP distribution of madIS, which doesn't contain the changes for MSPW. For MSPW to work, you'll need the head version of madIS from Hg. Clone it with: hg clone https://code.google.com/p/madis/ l.