Re: [pypy-dev] Speeding up zlib in standard library
(Replying on list - assuming Justin went off list my mistake) On Fri, Mar 16, 2012 at 4:54 AM, Justin Peel <peelpy@gmail.com> wrote:
Two things to mention. First, if you are going to use valgrind, you will need to build your own pypy because, as far as I know, the buildbot ones do not have the debug info so you won't have any helpful function names in your profile. If I remember correctly, most of the time used is in _operate in rzlib.py.
Also, in my opinion, the next thing to try in speeding up this code is to do a faster copy than a char by char copy for copying to the input buffer that is sent to the external C function. I'm not sure if the copying from the output buffer to the string builder is char by char or not.
Looking at the code for rzlib.py, _operate does seem to be the core function, and so likely the hot spot. https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rzlib.py It uses the StringBuilder class from rstring via the append_charpsize method, defined as follows: https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rstring.py def append_charpsize(self, s, size): l = [] for i in xrange(size): l.append(s[i]) self.l.append(self.tp("").join(l)) self._grow(size) So it is indeed doing a char by char copy of the string from Python to zlib (in my case to decompress a long chunk of data). I don't know enough about PyPy's internals to say if something naive like this would work faster (guessing looking at the append_slice method): def append_charpsize(self, s, size): assert 0 <= size self.l.append(s[0:size]) self._grow(size) Presumably to try this idea out I'm going to first need to get PyPy to build locally? Thanks, Peter
On Mon, Mar 19, 2012 at 5:49 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
(Replying on list - assuming Justin went off list my mistake)
On Fri, Mar 16, 2012 at 4:54 AM, Justin Peel <peelpy@gmail.com> wrote:
Two things to mention. First, if you are going to use valgrind, you will need to build your own pypy because, as far as I know, the buildbot ones do not have the debug info so you won't have any helpful function names in your profile. If I remember correctly, most of the time used is in _operate in rzlib.py.
Also, in my opinion, the next thing to try in speeding up this code is to do a faster copy than a char by char copy for copying to the input buffer that is sent to the external C function. I'm not sure if the copying from the output buffer to the string builder is char by char or not.
Looking at the code for rzlib.py, _operate does seem to be the core function, and so likely the hot spot. https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rzlib.py
It uses the StringBuilder class from rstring via the append_charpsize method, defined as follows: https://bitbucket.org/pypy/pypy/src/default/pypy/rlib/rstring.py
def append_charpsize(self, s, size): l = [] for i in xrange(size): l.append(s[i]) self.l.append(self.tp("").join(l)) self._grow(size)
So it is indeed doing a char by char copy of the string from Python to zlib (in my case to decompress a long chunk of data). I don't know enough about PyPy's internals to say if something naive like this would work faster (guessing looking at the append_slice method):
def append_charpsize(self, s, size): assert 0 <= size self.l.append(s[0:size]) self._grow(size)
Presumably to try this idea out I'm going to first need to get PyPy to build locally?
Thanks,
Peter _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
append_charpsize is special - it's not the *actual* implementation, the actual implementation is buried somewhere in rpython/lltypesystem/rbuilder.py, with the one you're mentioning being just fake implementation for tests. StringBuilder is special in a sense that it has some special GC support (which we can probably improve upon). Cheers, fijal
On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
append_charpsize is special - it's not the *actual* implementation, the actual implementation is buried somewhere in rpython/lltypesystem/rbuilder.py, with the one you're mentioning being just fake implementation for tests. StringBuilder is special in a sense that it has some special GC support (which we can probably improve upon).
Cheers, fijal
I guess you are referring to the copy_string_contents function here: https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.p... However, methods ll_append_multiple_char not ll_append_charpsize defined in rbuilder seem to use this - they both use a for loop char-by-char, https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuild... My hunch would be to replace this: @staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) for i in xrange(size): ll_builder.buf.chars[used] = charp[i] used += 1 ll_builder.used = used with this: @staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) assert size >= 0 ll_str.copy_contents(charp, ll_builder.buf, 0, used, size) ll_builder.used += size (and similarly for ll_append_multiple_char above it) Like an onion - more and more layers ;) I'm beginning to suspect speeding up append_charpsize in order to make passing strings to/from C code faster is a bit too ambitious for a first contribution to PyPy! [*] Peter [*] Especially as after three hours it is still building from source: $ python translate.py --opt=jit targetpypystandalone.py
On Mon, Mar 19, 2012 at 9:15 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
append_charpsize is special - it's not the *actual* implementation, the actual implementation is buried somewhere in rpython/lltypesystem/rbuilder.py, with the one you're mentioning being just fake implementation for tests. StringBuilder is special in a sense that it has some special GC support (which we can probably improve upon).
Cheers, fijal
I guess you are referring to the copy_string_contents function here: https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.p...
However, methods ll_append_multiple_char not ll_append_charpsize defined in rbuilder seem to use this - they both use a for loop char-by-char, https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuild...
My hunch would be to replace this:
@staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) for i in xrange(size): ll_builder.buf.chars[used] = charp[i] used += 1 ll_builder.used = used
with this:
@staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) assert size >= 0 ll_str.copy_contents(charp, ll_builder.buf, 0, used, size) ll_builder.used += size
(and similarly for ll_append_multiple_char above it)
Like an onion - more and more layers ;) I'm beginning to suspect speeding up append_charpsize in order to make passing strings to/from C code faster is a bit too ambitious for a first contribution to PyPy! [*]
Peter
[*] Especially as after three hours it is still building from source: $ python translate.py --opt=jit targetpypystandalone.py
ok, so let me reply a bit more :) First of all, you don't have to translate pypy to see changes. We mostly run tests to see if they work. You can also write a very small rpython program in translator/goal (look at targetnopstandalone.py) if you want to just test the performance of single function. I suppose your code is indeed a bit faster, but my bet would be it's not too much faster (feel free to prove me wrong, especially on older GCCs, they might not figure out that a loop is vectorizable for example). The main source of why passing strings to C is slow is however copying the string from the GC area to non-moving one, raw malloced in C. There are various strategies how to approach this, one of those would be pinning, so the GC structures don't move and you can pass a pointer to C. This is however definitely not a good first patch to pypy ;-) What I would suggest: * Your patch looks good to me, although I'm not sure if copy-string-contents would accept a raw memory. Check if tests pass. * If you want to benchmark, write a small test for passing such strings in translator/goal and see if it works. We're usually available for help on IRC and thanks for tackling this problem! Cheers, fijal
On Mon, Mar 19, 2012 at 2:32 PM, Maciej Fijalkowski <fijall@gmail.com>wrote:
On Mon, Mar 19, 2012 at 9:15 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Mon, Mar 19, 2012 at 6:36 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
append_charpsize is special - it's not the *actual* implementation, the actual implementation is buried somewhere in rpython/lltypesystem/rbuilder.py, with the one you're mentioning being just fake implementation for tests. StringBuilder is special in a sense that it has some special GC support (which we can probably improve upon).
Cheers, fijal
I guess you are referring to the copy_string_contents function here:
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rstr.p...
However, methods ll_append_multiple_char not ll_append_charpsize defined in rbuilder seem to use this - they both use a for loop
char-by-char,
https://bitbucket.org/pypy/pypy/src/default/pypy/rpython/lltypesystem/rbuild...
My hunch would be to replace this:
@staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) for i in xrange(size): ll_builder.buf.chars[used] = charp[i] used += 1 ll_builder.used = used
with this:
@staticmethod def ll_append_charpsize(ll_builder, charp, size): used = ll_builder.used if used + size > ll_builder.allocated: ll_builder.grow(ll_builder, size) assert size >= 0 ll_str.copy_contents(charp, ll_builder.buf, 0, used, size) ll_builder.used += size
(and similarly for ll_append_multiple_char above it)
Like an onion - more and more layers ;) I'm beginning to suspect speeding up append_charpsize in order to make passing strings to/from C code faster is a bit too ambitious for a first contribution to PyPy! [*]
Peter
[*] Especially as after three hours it is still building from source: $ python translate.py --opt=jit targetpypystandalone.py
ok, so let me reply a bit more :)
First of all, you don't have to translate pypy to see changes. We mostly run tests to see if they work. You can also write a very small rpython program in translator/goal (look at targetnopstandalone.py) if you want to just test the performance of single function.
I suppose your code is indeed a bit faster, but my bet would be it's not too much faster (feel free to prove me wrong, especially on older GCCs, they might not figure out that a loop is vectorizable for example).
The main source of why passing strings to C is slow is however copying the string from the GC area to non-moving one, raw malloced in C. There are various strategies how to approach this, one of those would be pinning, so the GC structures don't move and you can pass a pointer to C. This is however definitely not a good first patch to pypy ;-)
What I would suggest:
* Your patch looks good to me, although I'm not sure if copy-string-contents would accept a raw memory. Check if tests pass. * If you want to benchmark, write a small test for passing such strings in translator/goal and see if it works.
We're usually available for help on IRC and thanks for tackling this problem!
Cheers, fijal _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
FWIW, copy_string_contents definitely doesn't take raw memory, it takes an rstr. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero
Hi again, Based on the structure of pypy/translator/goal/richards.py and pypy/translator/goal/targetrichards.py I have tried to make a simple ZLIB based benchmark based on what I have been using. $ python2.6 green_bottles.py Green bottles ZLIB benchmark starting... [<function entry_point at 0x1004c7500>] Trying ZLIB on 176683 bytes of text 10000 times finished. Total time for 10000 iterations: 34.25 secs Average time per iteration: 3.42 ms $ pypy green_bottles.py Green bottles ZLIB benchmark starting... [<function entry_point at 0x0000000102f3b3d0>] Trying ZLIB on 176683 bytes of text 10000 times finished. Total time for 10000 iterations: 45.12 secs Average time per iteration: 4.51 ms Here PyPy v1.8 is about 1.3 times as slow as C Python 2.6, not as bad as the factor of almost two I was seeing in some cases, but enough to tackle as a measurable target? Files here: https://gist.github.com/2136181 However, I have a problem with using translate.py on this (trying the PyPy release-1.8 tag or the latest from bitbucket): $ python2.6 translate.py --run targetzlib.py [platform:msg] Setting platform to 'host' cc=None [translation:info] Translating target as defined by targetzlib [platform:execute] gcc ... [translation] translate.py configuration: [translation] [translate] [translation] [goal_options] [translation] run = True [translation] targetspec = targetzlib [translation] translation configuration: [translation] [translation] [translation] gc = minimark [translation] gctransformer = framework [translation] list_comprehension_operations = True [translation] withsmallfuncsets = 5 [translation:info] Annotating&simplifying... [translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy [platform:execute] gcc ... [Timer] Timings: [Timer] annotate --- 1.6 s [Timer] ======================================== [Timer] Total: --- 1.6 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 309, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do [translation:ERROR] res = func() [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 324, in task_annotate [translation:ERROR] s = annotator.build_types(self.entry_point, self.inputtypes) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 107, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, inputcells, complete_now=complete_now) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 198, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 254, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 452, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 512, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 695, in consider_op [translation:ERROR] argcells = [self.binding(a) for a in op.args] [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 297, in binding [translation:ERROR] return self.bookkeeper.immutableconstant(arg) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 318, in immutableconstant [translation:ERROR] return self.immutablevalue(const.value) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 469, in immutablevalue [translation:ERROR] result = SomePBC([self.getdesc(x)]) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 535, in getdesc [translation:ERROR] pyobj,)) [translation:ERROR] Exception: unexpected prebuilt constant: <built-in function compressobj> [translation:ERROR] Processing block: [translation:ERROR] block@27 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.translator.goal.green_bottles:10)compress [translation:ERROR] containing the following operations: [translation:ERROR] v0 = simple_call((builtin_function_or_method compressobj), (6), (8), (-15), (8), (0)) [translation:ERROR] v1 = getattr(v0, ('compress')) [translation:ERROR] v2 = simple_call(v1, orig_data_0) [translation:ERROR] v3 = getattr(v0, ('flush')) [translation:ERROR] v4 = simple_call(v3) [translation:ERROR] v5 = add(v2, v4) [translation:ERROR] v6 = simple_call((builtin_function_or_method crc32), orig_data_0) [translation:ERROR] v7 = newtuple(v5, v6) [translation:ERROR] --end-- [translation] start debugger...
/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py(535)getdesc() -> pyobj,)) (Pdb+) quit
Calling it without the --run gives the same: $ python2.6 translate.py targetzlib.py ... Should my benchmark Python code be RPython? Thanks, Peter
On Tue, Mar 20, 2012 at 4:40 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
Hi again,
Based on the structure of pypy/translator/goal/richards.py and pypy/translator/goal/targetrichards.py I have tried to make a simple ZLIB based benchmark based on what I have been using.
$ python2.6 green_bottles.py Green bottles ZLIB benchmark starting... [<function entry_point at 0x1004c7500>] Trying ZLIB on 176683 bytes of text 10000 times finished. Total time for 10000 iterations: 34.25 secs Average time per iteration: 3.42 ms
$ pypy green_bottles.py Green bottles ZLIB benchmark starting... [<function entry_point at 0x0000000102f3b3d0>] Trying ZLIB on 176683 bytes of text 10000 times finished. Total time for 10000 iterations: 45.12 secs Average time per iteration: 4.51 ms
Here PyPy v1.8 is about 1.3 times as slow as C Python 2.6, not as bad as the factor of almost two I was seeing in some cases, but enough to tackle as a measurable target?
Files here: https://gist.github.com/2136181
However, I have a problem with using translate.py on this (trying the PyPy release-1.8 tag or the latest from bitbucket):
$ python2.6 translate.py --run targetzlib.py [platform:msg] Setting platform to 'host' cc=None [translation:info] Translating target as defined by targetzlib [platform:execute] gcc ... [translation] translate.py configuration: [translation] [translate] [translation] [goal_options] [translation] run = True [translation] targetspec = targetzlib [translation] translation configuration: [translation] [translation] [translation] gc = minimark [translation] gctransformer = framework [translation] list_comprehension_operations = True [translation] withsmallfuncsets = 5 [translation:info] Annotating&simplifying... [translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy [platform:execute] gcc ... [Timer] Timings: [Timer] annotate --- 1.6 s [Timer] ======================================== [Timer] Total: --- 1.6 s [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "translate.py", line 309, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do [translation:ERROR] res = func() [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 324, in task_annotate [translation:ERROR] s = annotator.build_types(self.entry_point, self.inputtypes) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 107, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, inputcells, complete_now=complete_now) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 198, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 254, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 452, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 512, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 695, in consider_op [translation:ERROR] argcells = [self.binding(a) for a in op.args] [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 297, in binding [translation:ERROR] return self.bookkeeper.immutableconstant(arg) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 318, in immutableconstant [translation:ERROR] return self.immutablevalue(const.value) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 469, in immutablevalue [translation:ERROR] result = SomePBC([self.getdesc(x)]) [translation:ERROR] File "/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 535, in getdesc [translation:ERROR] pyobj,)) [translation:ERROR] Exception: unexpected prebuilt constant: <built-in function compressobj> [translation:ERROR] Processing block: [translation:ERROR] block@27 is a <class 'pypy.objspace.flow.flowcontext.SpamBlock'> [translation:ERROR] in (pypy.translator.goal.green_bottles:10)compress [translation:ERROR] containing the following operations: [translation:ERROR] v0 = simple_call((builtin_function_or_method compressobj), (6), (8), (-15), (8), (0)) [translation:ERROR] v1 = getattr(v0, ('compress')) [translation:ERROR] v2 = simple_call(v1, orig_data_0) [translation:ERROR] v3 = getattr(v0, ('flush')) [translation:ERROR] v4 = simple_call(v3) [translation:ERROR] v5 = add(v2, v4) [translation:ERROR] v6 = simple_call((builtin_function_or_method crc32), orig_data_0) [translation:ERROR] v7 = newtuple(v5, v6) [translation:ERROR] --end-- [translation] start debugger...
/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py(535)getdesc() -> pyobj,)) (Pdb+) quit
Calling it without the --run gives the same:
$ python2.6 translate.py targetzlib.py ...
Should my benchmark Python code be RPython?
Thanks,
Peter _______________________________________________ pypy-dev mailing list pypy-dev@python.org http://mail.python.org/mailman/listinfo/pypy-dev
Hey You're confusing levels. Your benchmark code cannot call zlib. I suggested you benchmark not zlib, but your changes to StringBuilder that you proposed. That does not (and cannot) require zlib. You'll be testing something smaller that you're trying to change instead. Then obviously a comparison with CPython is meaningless. Cheers, fijal
On Tue, Mar 20, 2012 at 2:52 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hey
You're confusing levels. Your benchmark code cannot call zlib.
PyPy development has quite a learning curve.
I suggested you benchmark not zlib, but your changes to StringBuilder that you proposed. That does not (and cannot) require zlib. You'll be testing something smaller that you're trying to change instead. Then obviously a comparison with CPython is meaningless.
i.e. I would create the new StringBuilder benchmark as the file pypy/translator/targetStringBuilder.py and do this: $ cd pypy/translator/goal/ $ python translate.py targetStringBuilder.py $ ./targetStringBuilder-c Then modify the pypy/rpython/lltypesystem/rbuilder.py (or whatever) and repeat this. And that should compare the benchmark translation by the virgin PyPy against the translation by my modified PyPy? That doesn't sound quite so intimidating... and if I've understood this now it does seem like the basis of a good development FAQ entry: http://doc.pypy.org/en/latest/faq.html#development Then if the benchmark results are encouraging, in principle I could then recompile the whole of pypy (which is slow), and then go back to run the patched pypy on my real script to see what difference it makes, if any. Right? Thanks for bearing with me, Peter
On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Tue, Mar 20, 2012 at 2:52 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
Hey
You're confusing levels. Your benchmark code cannot call zlib.
PyPy development has quite a learning curve.
I suggested you benchmark not zlib, but your changes to StringBuilder that you proposed. That does not (and cannot) require zlib. You'll be testing something smaller that you're trying to change instead. Then obviously a comparison with CPython is meaningless.
i.e. I would create the new StringBuilder benchmark as the file pypy/translator/targetStringBuilder.py and do this:
$ cd pypy/translator/goal/ $ python translate.py targetStringBuilder.py $ ./targetStringBuilder-c
Then modify the pypy/rpython/lltypesystem/rbuilder.py (or whatever) and repeat this. And that should compare the benchmark translation by the virgin PyPy against the translation by my modified PyPy?
That doesn't sound quite so intimidating... and if I've understood this now it does seem like the basis of a good development FAQ entry: http://doc.pypy.org/en/latest/faq.html#development
Then if the benchmark results are encouraging, in principle I could then recompile the whole of pypy (which is slow), and then go back to run the patched pypy on my real script to see what difference it makes, if any.
Right?
Thanks for bearing with me,
Peter
Right, you got it perfect :) Can you suggest the wording for a FAQ entry? Cheers, fijal
On Wed, Mar 21, 2012 at 5:49 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
That doesn't sound quite so intimidating... and if I've understood this now it does seem like the basis of a good development FAQ entry: http://doc.pypy.org/en/latest/faq.html#development
Then if the benchmark results are encouraging, in principle I could then recompile the whole of pypy (which is slow), and then go back to run the patched pypy on my real script to see what difference it makes, if any.
Right?
Thanks for bearing with me,
Peter
Right, you got it perfect :)
Can you suggest the wording for a FAQ entry?
Cheers, fijal
Something like this? Peter. Q: How can I test and benchmark a modification to PyPy, for example in a StringBuilder method? A: Based on the existing examples, create a new StringBuilder benchmark as the file pypy/translator/targetStringBuilder.py which will verify the functionality and/or time it, and do this: $ cd pypy/translator/goal/ $ python translate.py targetStringBuilder.py $ ./targetStringBuilder-c Then make your modifications to pypy/rpython/lltypesystem/rbuilder.py (or whatever you are working on) and repeat this. That should compare the benchmark translation by the virgin PyPy against the translation by your modified PyPy. Once you have a potentially useful improvement, you should run the full test suite to ensure there are no side effects. Q: How do I run PyPy's unit tests? A: ...
On Tue, Mar 27, 2012 at 3:27 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
On Wed, Mar 21, 2012 at 5:49 PM, Maciej Fijalkowski <fijall@gmail.com> wrote:
On Wed, Mar 21, 2012 at 7:10 PM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
That doesn't sound quite so intimidating... and if I've understood this now it does seem like the basis of a good development FAQ entry: http://doc.pypy.org/en/latest/faq.html#development
Then if the benchmark results are encouraging, in principle I could then recompile the whole of pypy (which is slow), and then go back to run the patched pypy on my real script to see what difference it makes, if any.
Right?
Thanks for bearing with me,
Peter
Right, you got it perfect :)
Can you suggest the wording for a FAQ entry?
Cheers, fijal
Something like this? Peter.
Q: How can I test and benchmark a modification to PyPy, for example in a StringBuilder method?
A: Based on the existing examples, create a new StringBuilder benchmark as the file pypy/translator/targetStringBuilder.py which will verify the functionality and/or time it, and do this:
$ cd pypy/translator/goal/ $ python translate.py targetStringBuilder.py $ ./targetStringBuilder-c
Then make your modifications to pypy/rpython/lltypesystem/rbuilder.py (or whatever you are working on) and repeat this. That should compare the benchmark translation by the virgin PyPy against the translation by your modified PyPy. Once you have a potentially useful improvement, you should run the full test suite to ensure there are no side effects.
Q: How do I run PyPy's unit tests?
A: ...
This sounds overly specific to me. What do others think?
Hi Fijal, On Tue, Mar 27, 2012 at 15:30, Maciej Fijalkowski <fijall@gmail.com> wrote:
This sounds overly specific to me. What do others think?
It is indeed overly specific, but it may be useful nevertheless. We need to rephrase it to point out the specific-vs-general parts. Something like this: Q: How can I test and benchmark a modification to RPython? (As opposed to a modification done in the source code of the PyPy Python interpreter) A: As an example, let's say that you want to tweak pypy.rlib.rstring.StringBuilder. This file contains the implementation for tests only; the real translated implementation is in pypy.rpython.lltypesystem.rbuilder. This is tested by pypy.rpython.test.test_rbuilder. Be sure that any tweak you do still passes the existing tests, and if possible, add new tests specifically for your changes. (Run the tests with: python test_all.py rpython/test/test_rbuilder.py) Then to get benchmarks: based on the existing examples, create a new StringBuilder benchmark as the file pypy/translator/targetStringBuilder.py which will time the functionality --- written as a small RPython program --- and do this: $ cd pypy/translator/goal/ $ python translate.py targetStringBuilder.py $ ./targetStringBuilder-c You don't need to translate the full PyPy Python interpreter to benchmark every change. However, you should do it once at the end, to make sure that no corner case has been missed and that the performance improvements are visible there as well. --- Armin
participants (4)
-
Alex Gaynor -
Armin Rigo -
Maciej Fijalkowski -
Peter Cock