[pypy-dev] I was talking with Russel Winder at PyCON UK.

[Josh Ayers]

I don't think it's due to the warmup of the JIT.  Here's a simpler example.

import time
import multiprocessing

def do_nothing(): pass

if __name__ == '__main__':
   time1 = time.time()
   do_nothing()
   time2 = time.time()
   pool = multiprocessing.Pool(processes=1)
   time3 = time.time()
   result = pool.apply_async(do_nothing)
   result.get()
   time4 = time.time()
   result = pool.apply_async(do_nothing)
   result.get()
   time5 = time.time()
   pool.close()

   print('not multiprocessing: ' + str(time2 - time1))
   print('create pool: ' + str(time3 - time2))
   print('run first time: ' + str(time4 - time3))
   print('run second time: ' + str(time5 - time4))

Here are the results in PyPy.  The first call to do_nothing() using
multiprocessing.Pool takes 0.57 seconds.

not multiprocessing: 0.0
create pool: 0.30999994278
run first time: 0.575999975204
run second time: 0.00100016593933

Here are the results in CPython.  It also appears to be have some overhead
the first time the pool is used, but it's less severe than PyPy.

not multiprocessing: 0.0
create pool: 0.00500011444092
run first time: 0.134000062943
run second time: 0.0



On Fri, Sep 30, 2011 at 6:25 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> On Fri, Sep 30, 2011 at 10:20 AM, Armin Rigo <arigo at tunes.org> wrote:
> > Hi,
> >
> > Is the conclusion just the fact that, again, the JIT's warm-up time is
> > important, which we know very well?  Or is there some other effect
> > that cannot be explained just by that?  (BTW, Laura, it's unrelated to
> > multithreading if it's based on the multiprocessing module.)
> >
>
> I guess what people didn't realize is that if you spawn a new process,
> you have to warmup the JIT *again* for each of the worker (at least in
> the worst case scenario).
>
> >
> > A bientôt,
> >
> > Armin.
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110930/6ae462b7/attachment-0001.html>