[pypy-dev] I was talking with Russel Winder at PyCON UK.
Josh Ayers
josh.ayers at gmail.com
Fri Sep 30 05:24:17 CEST 2011
I think the slowdown you're seeing is due to the time it takes to create new
processes. This seems to be quite a bit slower in PyPy than in CPython.
However, once the process pool is created and has been used once, the
execution time vs. process count behaves as expected.
I attached a modified version of your code to demonstrate the behavior. It
calculates Pi once without using multiprocessing as a baseline for
comparison. Then a multiprocessing.Pool object is created with 8 processes,
and the same pool is used multiple times. On my machine, creating the 8 new
processes takes 0.60 seconds in PyPy and only 0.20 seconds in CPython.
The pool is first used two times in a row with only a single process
active. For some reason, the second run is a factor of 2 faster than the
first. Is this just warmup of the JIT, or some other behavior?
Next, it repeats using 2, 4, and 8 processes. This was run on a 4 core
machine, and as expected there was an improvement in run time with 2 and 4
processes. Using 8 processes gives approximately the same run time as 4.
The output is pasted below. I also pasted the modified code here in case
the attached file doesn't come through: http://pastie.org/2614751. For
reference, I'm running PyPy 1.6 on Windows 7.
Sincerely,
Josh
C:\Users\jayers\Documents\SVN\randomStuff\pypy_comparisons>pypy-c
pi_python2_multiprocessing_pool.py
3.14159265359
non parallel execution time: 1.52899980545
pool creation time: 0.559000015259
==== Python Multiprocessing Pool pi = 3.14159265359
==== Python Multiprocessing Pool iteration count = 100000000
==== Python Multiprocessing Pool elapse = 3.1930000782
==== Python Multiprocessing Pool process count = 1
==== Python Multiprocessing Pool processor count = 4
==== Python Multiprocessing Pool pi = 3.14159265359
==== Python Multiprocessing Pool iteration count = 100000000
==== Python Multiprocessing Pool elapse = 1.53900003433
==== Python Multiprocessing Pool process count = 1
==== Python Multiprocessing Pool processor count = 4
==== Python Multiprocessing Pool pi = 3.14159265359
==== Python Multiprocessing Pool iteration count = 100000000
==== Python Multiprocessing Pool elapse = 0.802000045776
==== Python Multiprocessing Pool process count = 2
==== Python Multiprocessing Pool processor count = 4
==== Python Multiprocessing Pool pi = 3.14159265359
==== Python Multiprocessing Pool iteration count = 100000000
==== Python Multiprocessing Pool elapse = 0.441999912262
==== Python Multiprocessing Pool process count = 4
==== Python Multiprocessing Pool processor count = 4
==== Python Multiprocessing Pool pi = 3.14159265359
==== Python Multiprocessing Pool iteration count = 100000000
==== Python Multiprocessing Pool elapse = 0.457000017166
==== Python Multiprocessing Pool process count = 8
==== Python Multiprocessing Pool processor count = 4
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110929/2d13a8b5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pi_python2_multiprocessing_pool.py
Type: application/octet-stream
Size: 1604 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110929/2d13a8b5/attachment-0001.obj>
More information about the pypy-dev
mailing list