Here's a further modified version. In this case, when using the pool for the first time, it uses an n of 10, instead of 100 million. Even with such a low precision, the first execution takes 1.3 seconds. It seems some significant warm up time is needed the first time a multiprocessing.Pool object is used.<br>
<br>See the attachment or this link for the code: <a href="http://pastie.org/2614925">http://pastie.org/2614925</a><br><br><br><br><div class="gmail_quote">On Thu, Sep 29, 2011 at 8:24 PM, Josh Ayers <span dir="ltr"><<a href="mailto:josh.ayers@gmail.com">josh.ayers@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I think the slowdown you're seeing is due to the time it takes to create new processes. This seems to be quite a bit slower in PyPy than in CPython. However, once the process pool is created and has been used once, the execution time vs. process count behaves as expected.<br>
<br>I attached a modified version of your code to demonstrate the behavior. It calculates Pi once without using multiprocessing as a baseline for comparison. Then a multiprocessing.Pool object is created with 8 processes, and the same pool is used multiple times. On my machine, creating the 8 new processes takes 0.60 seconds in PyPy and only 0.20 seconds in CPython.<br>
<br>The pool is first used two times in a row with only a single process active. For some reason, the second run is a factor of 2 faster than the first. Is this just warmup of the JIT, or some other behavior?<br><br>Next, it repeats using 2, 4, and 8 processes. This was run on a 4 core machine, and as expected there was an improvement in run time with 2 and 4 processes. Using 8 processes gives approximately the same run time as 4.<br>
<br>The output is pasted below. I also pasted the modified code here in case the attached file doesn't come through: <a href="http://pastie.org/2614751" target="_blank">http://pastie.org/2614751</a>. For reference, I'm running PyPy 1.6 on Windows 7.<br>
<br>Sincerely,<br>Josh<br><br><br>C:\Users\jayers\Documents\SVN\randomStuff\pypy_comparisons>pypy-c pi_python2_multiprocessing_pool.py<br>
<br>3.14159265359<br>non parallel execution time: 1.52899980545<br>pool creation time: 0.559000015259<br>==== Python Multiprocessing Pool pi = 3.14159265359<br>==== Python Multiprocessing Pool iteration count = 100000000<br>
==== Python Multiprocessing Pool elapse = 3.1930000782<br>==== Python Multiprocessing Pool process count = 1<br>==== Python Multiprocessing Pool processor count = 4<br><br><br>==== Python Multiprocessing Pool pi = 3.14159265359<br>
==== Python Multiprocessing Pool iteration count = 100000000<br>==== Python Multiprocessing Pool elapse = 1.53900003433<br>==== Python Multiprocessing Pool process count = 1<br>==== Python Multiprocessing Pool processor count = 4<br>
<br><br>==== Python Multiprocessing Pool pi = 3.14159265359<br>==== Python Multiprocessing Pool iteration count = 100000000<br>==== Python Multiprocessing Pool elapse = 0.802000045776<br>==== Python Multiprocessing Pool process count = 2<br>
==== Python Multiprocessing Pool processor count = 4<br><br><br>==== Python Multiprocessing Pool pi = 3.14159265359<br>==== Python Multiprocessing Pool iteration count = 100000000<br>==== Python Multiprocessing Pool elapse = 0.441999912262<br>
==== Python Multiprocessing Pool process count = 4<br>==== Python Multiprocessing Pool processor count = 4<br><br><br>==== Python Multiprocessing Pool pi = 3.14159265359<br>==== Python Multiprocessing Pool iteration count = 100000000<br>
==== Python Multiprocessing Pool elapse = 0.457000017166<br>==== Python Multiprocessing Pool process count = 8<br>==== Python Multiprocessing Pool processor count = 4<br><br><br>
</blockquote></div><br>