[pypy-issue] [issue884] Diabolical Multicore Thread Performance

David Beazley tracker at bugs.pypy.org
Wed Sep 28 04:14:21 CEST 2011

New submission from David Beazley <dave at dabeaz.com>:

This problem is pretty easy to recreate.   I'm using pypy-1.6.

1. Go find a multicore machine with 4 or more cores (e.g., a 4-core MacBook Pro,
or 8-core Amazon EC2 Linux instance).

2. Launch pypy and spin up a single CPU-bound thread.

>>>> def spinner()
....     n = 0
....     while True:
....         n += 1
>>>> import threading
>>>> t = threading.Thread(target=spinner)
>>>> t.daemon=True
>>>> t.start()

3. Sit back and observe that the interactive interpreter (and just about
everything else related to I/O) is so slow as to be nearly unresponsive (try
doing some things at the prompt).

The responsiveness gets significantly worse if the number of cores is increased.
 For example, I've been running some socket I/O benchmarks where I/O is mixed
with a *single* CPU-bound thread.  If there is a single CPU core, code runs in
about 0.05s.  With 4 cores+hyperthreading, the *same* test runs in 730s (a
slowdown of about 14600x).   The specifics of this test aren't so
interesting--you can easily observe a huge slowdown just by launching a thread
and trying to use the interactive prompt.

What's the cause?  I have no idea except that my machine does not appear to be
wildly thrashing (e.g., having a GIL battle).  Good luck ;-).

assignedto: agaynor
messages: 3201
nosy: agaynor, dabeaz, pypy-issue
priority: bug
release: ???
status: unread
title: Diabolical Multicore Thread Performance

PyPy bug tracker <tracker at bugs.pypy.org>

More information about the pypy-issue mailing list