[pypy-dev] Need help in concluding Multi-Thread based model thoughts between Python and PyPy

[Sasikanth Eda]

Hi All,

I have tried to perform a CPU intensive operation ( which typically
involved calculation of prime numbers from 1-100000 ) using Multi-process
and Multi-Thread models provided by
C, Python-2.6, PyPy-1.9.

Here are results that are obtained from this exercise;

*C Program results ( Number of processes = 500 )*

CPU utilization      = 99.81% (peak)
Memory utilization = 14.77 GB (peak)
Execution time      = *2m16s* (real) ( time taken between invocation to
termination )

*C Program results ( Number of Threads = 500 )*

CPU utilization      = 99.94% (peak)
Memory utilization = 14.79 GB (peak)
Execution time      = *2m10s* (real)

*{ Here I came to a conclusion that C gave a comparatively equal
good results in terms of both Multi-Threaded as well as Multi-Process }*

*Python Program results ( Number of processes = 500 )*

CPU utilization      = 99.62% (peak)
Memory utilization = 17.03 GB (peak)
Execution time      = *87m42.785s* (real)

*Python Program results ( Number of Threads = 500 )*

CPU utilization      = 18.23% (peak)
Memory utilization = 33.074 GB (peak)
Execution time      = *239m10.210s* (real)

*{ Here I came to a conclusion that Python is limited by the Global
Interpreter Look (GIL) when used in Multi-Thread mode (that is why it gave
poor results when compared with process model }*

*PyPy Program results ( Number of processes = 500 )*

CPU utilization      = 99.19% (peak)
Memory utilization = 22.92 GB (peak)
Execution time      = *6m4.970s* (real)

*PyPy Program results ( Number of Threads = 500 )*

CPU utilization      = 38.88% (peak)
Memory utilization = 21.06 GB (peak)
Execution time      = *59m29s* (real)

*{ Here I came to a conclusion that PyPy is better than Python }*
*
*
Now what am I struggling to understand is ;
*
*
1.  Has PyPy optimized / reduced the GIL limitation ? ( what is the
progress in PyPy version 1.9 in that when compared with Python's progress )
2.  If PyPy is also suffering from the same GIL limitations, what made the
program run faster than Python, is it because of
more warm-up time, optimization of loops  ?

3.  What are your suggestions for me if I wanted to go for
Multi-Thread application design ( in-terms of Python / PyPy )

I request your inputs/suggestion techniques towards
optimizing Multi threaded load loads.

Kindly share your experiences and feedback in this scenario.

-- 
Sasikanth
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20121030/f6633640/attachment.html>