[pypy-dev] Game state search dominated by copy.deepcopy

[Maciej Fijalkowski]

I don't think pypy is expected to speed up program where the majority
of the time is spent in copy.deepcopy. The answer to this question is
a bit boring one: don't write algorithms that copy around so much
data.

As a general rule - if the vast majority of your work is done by the
runtime there is no way for pypy to speed this up (mostly) and in fact
the exact same algo in C would be as slow or slower.

I'm sorry we can't give you a very good answer - if you really *NEED*
to do that much copying (maybe you can make the copy more lazy?), then
maybe shrinking the data structures would help? I can't answer that
without having access to the code though, which I would be happy to
look at.

Cheers,
fijal

On Sat, May 14, 2016 at 1:19 AM, Eli Stevens (Gmail)
<wickedgrey at gmail.com> wrote:
> Hello,
>
> I'm in the process of working on a hobby project to have an AI
> searching through a game state space. I recently ran what I have so
> far on pypy (I had been doing initial work on cpython), and got two
> results that were unexpected:
>
> - The total execution time was basically identical between cpython and pypy
> - The runtime on both pythons was about 50% copy.deepcopy (called on
> the main game state object)
>
> The runtime of the script that I've been using is in the 30s to 2m
> range, depending on config details, and the implementation fits the
> following pretty well:
>
> """
>  The JIT is generally good at speeding up straight-forward Python code
> that spends a lot of time in the bytecode dispatch loop, i.e., running
> actual Python code – as opposed to running things that only are
> invoked by Python code. Good examples include numeric calculations or
> any kind of heavily object-oriented program.
> """
>
> I have already pulled out constant game information (static info like
> unit stats, etc.) into an object that isn't copied, and a lot of the
> numerical data that is copied is stored in a numpy array so that I
> don't have hundreds of dicts, ints, etc.
>
> First, is there a good way to speed up object copying? I've tried
> pickling to a cache, and unpickling from there (so as to only pickle
> once), but that didn't make a significant difference.
>
> http://vmprof.com/#/905dfb71d28626bff6341a5848deae73 (deepcopy)
> http://vmprof.com/#/545f1243b345eb9e41d73a9043a85efd (pickle)
>
> Second, what's the best way to start figuring out why pypy isn't able
> to outperform cpython on my program?
>
> Thanks for any pointers,
> Eli
> _______________________________________________
> pypy-dev mailing list
> pypy-dev at python.org
> https://mail.python.org/mailman/listinfo/pypy-dev