[pypy-dev] Speeding up zlib in standard library

[Peter Cock]

Hi again,

Based on the structure of pypy/translator/goal/richards.py and
pypy/translator/goal/targetrichards.py I have tried to make a
simple ZLIB based benchmark based on what I have been using.

$ python2.6 green_bottles.py
Green bottles ZLIB benchmark starting... [<function entry_point at 0x1004c7500>]
Trying ZLIB on 176683 bytes of text 10000 times
finished.
Total time for 10000 iterations: 34.25 secs
Average time per iteration: 3.42 ms

$ pypy green_bottles.py
Green bottles ZLIB benchmark starting... [<function entry_point at
0x0000000102f3b3d0>]
Trying ZLIB on 176683 bytes of text 10000 times
finished.
Total time for 10000 iterations: 45.12 secs
Average time per iteration: 4.51 ms

Here PyPy v1.8 is about 1.3 times as slow as C Python 2.6, not
as bad as the factor of almost two I was seeing in some cases,
but enough to tackle as a measurable target?

Files here: https://gist.github.com/2136181

However, I have a problem with using translate.py on this
(trying the PyPy release-1.8 tag or the latest from bitbucket):

$ python2.6 translate.py --run targetzlib.py
[platform:msg] Setting platform to 'host' cc=None
[translation:info] Translating target as defined by targetzlib
[platform:execute] gcc ...
[translation] translate.py configuration:
[translation] [translate]
[translation]     [goal_options]
[translation]         run = True
[translation]     targetspec = targetzlib
[translation] translation configuration:
[translation] [translation]
[translation]     gc = minimark
[translation]     gctransformer = framework
[translation]     list_comprehension_operations = True
[translation]     withsmallfuncsets = 5
[translation:info] Annotating&simplifying...
[translation:info] with policy: pypy.annotation.policy.AnnotatorPolicy
[platform:execute] gcc ...
[Timer] Timings:
[Timer] annotate                       --- 1.6 s
[Timer] ========================================
[Timer] Total:                         --- 1.6 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "translate.py", line 309, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 810, in
proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/tool/taskengine.py", line 116,
in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 287, in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/translator/driver.py", line 324, in
task_annotate
[translation:ERROR]     s = annotator.build_types(self.entry_point,
self.inputtypes)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 107, in
build_types
[translation:ERROR]     return self.build_graph_types(flowgraph,
inputcells, complete_now=complete_now)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 198, in
build_graph_types
[translation:ERROR]     self.complete()
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 254, in
complete
[translation:ERROR]     self.processblock(graph, block)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 452, in
processblock
[translation:ERROR]     self.flowin(graph, block)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 512, in
flowin
[translation:ERROR]     self.consider_op(block.operations[i])
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 695, in
consider_op
[translation:ERROR]     argcells = [self.binding(a) for a in op.args]
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/annrpython.py", line 297, in
binding
[translation:ERROR]     return self.bookkeeper.immutableconstant(arg)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 318, in
immutableconstant
[translation:ERROR]     return self.immutablevalue(const.value)
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 469, in
immutablevalue
[translation:ERROR]     result = SomePBC([self.getdesc(x)])
[translation:ERROR]    File
"/Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py", line 535, in
getdesc
[translation:ERROR]     pyobj,))
[translation:ERROR]  Exception: unexpected prebuilt constant:
<built-in function compressobj>
[translation:ERROR] Processing block:
[translation:ERROR]  block at 27 is a <class
'pypy.objspace.flow.flowcontext.SpamBlock'>
[translation:ERROR]  in (pypy.translator.goal.green_bottles:10)compress
[translation:ERROR]  containing the following operations:
[translation:ERROR]        v0 =
simple_call((builtin_function_or_method compressobj), (6), (8), (-15),
(8), (0))
[translation:ERROR]        v1 = getattr(v0, ('compress'))
[translation:ERROR]        v2 = simple_call(v1, orig_data_0)
[translation:ERROR]        v3 = getattr(v0, ('flush'))
[translation:ERROR]        v4 = simple_call(v3)
[translation:ERROR]        v5 = add(v2, v4)
[translation:ERROR]        v6 =
simple_call((builtin_function_or_method crc32), orig_data_0)
[translation:ERROR]        v7 = newtuple(v5, v6)
[translation:ERROR]  --end--
[translation] start debugger...
> /Users/pjcock/pypy-hg/pypy/annotation/bookkeeper.py(535)getdesc()
-> pyobj,))
(Pdb+) quit

Calling it without the --run gives the same:

$ python2.6 translate.py targetzlib.py
...

Should my benchmark Python code be RPython?

Thanks,

Peter