[pypy-dev] Efficiently piecing together callables with hooks?

[Samuel Ytterbrink]

Hi, i only have some questions about your example code. How does the
runhooks function work? And what are you using the pos loop variable for?
Den 14 aug 2011 18.58 skrev "Timo Paulssen" <timonator at perpetuum-immobile.de
>:
> Hello List,
>
> I'm currently experimenting with a piece of software I've recently started
> maintaining, which is a cellular automaton simulation software. It's
supposed to
> make experimenting easy, but still be fast at simulating. As you might
know, all
> cellular automatons operate in steps, where usually the whole
"configuration
> space" is iterated over and all fields are updated depending on their
> neighbourhood and the "local transition function". Currently, the step
functions
> are implemented in C and inlined with weave.inline.
>
> The experiment I just tried out was generating step functions by piecing
> together strings of C code. It was quite successful, but I would like to
take
> that one step further and use python code instead of having the user write
C
> code that has to be written with weave in mind.
>
> The first attempt at making it work with python code was to put python
callables
> into lists and running them somewhat like this:
>
> def step_pure_py(self):
> runhooks("init")
> loop_iter = self.loop.get_iter()
> for pos in loop_iter:
> runhooks("pre_compute")
> runhooks("compute")
> runhooks("post_compute")
> runhooks("after_step")
>
> This is utterly slow, because - even though i turned the lists that
contain the
> hooks into tuples - i have no way to tell pypy, that the lists are not
going to
> be changed and all loops that handle them can safely be unrolled.
>
> Unrolling the lists by hand gives me insane speedups. These are timings
for
> 10.000 steps on a 1000 field array - that is calling step_pure_py 1000
times and
> having the loop_iter iterate over 1000 fields each time.
>
> hand-unrolled python code:
> 2.15user 0.01system 0:02.18elapsed 99%CPU (0avgtext+0avgdata
103024maxresident)k
>
> hook based solution like above:
> 62.57user 0.05system 1:02.79elapsed 99%CPU (0avgtext+0avgdata
184576maxresident)k
>
> So what would you suggest for making this much faster? I read on the IRC,
that
> in pypy itself, string concatenation is actually used to create python
> functions, so maybe that would be way to go in this case? Or could pypy
offer
> decorators or other hints to cause pypy to unroll and constant-fold
different
> things for me?
>
> Thank you
> - Timo Paulssen
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> http://mail.python.org/mailman/listinfo/pypy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110815/1641dbcc/attachment.html>