[New-bugs-announce] [issue26360] Deadlock in thread.join

[Mark Dickinson]

New submission from Mark Dickinson:

On OS X (10.9.5), I'm getting an apparent deadlock in the following simple Python script:

#--------------------------------------------------------
import itertools
import threading

def is_prime(n):
    return n >= 2 and all(n % d for d in xrange(2, n))

def count_primes_in_range(start, stop):
    return sum(is_prime(n) for n in xrange(start, stop))

def main():
    threads = [
        threading.Thread(
            target=count_primes_in_range,
            args=(12500*i, 12500*(i+1))
        )
        for i in xrange(8)
    ]
    for thread in threads:
        thread.start()
    for thread in threads:
        thread.join()


if __name__ == '__main__':
    for i in itertools.count():
        print "Iteration: ", i
        main()
#--------------------------------------------------------

Each iteration takes around 60 seconds, and I typically get a deadlock within the first 5 iterations. It looks as though the deadlock happens during the "thread.join", at a stage where some of the threads have already completed and been joined. The code hangs with no CPU activity, and a backtrace (attached) shows that all the background threads are waiting for the GIL, while the main thread is in a blocking `thread.join` call.

I've attached a gdb-generated stack trace.

I was unable to reproduce this with a debug build of Python. I *have* reproduced with a normal build of Python, and on various Python 2.7 executables from 3rd party sources (Apple, Macports, Enthought Canopy).

I've also not yet managed to reproduce on Python 3, but I haven't tried that hard. I suspect it's a Python 2-only problem, though.

(And yes, this is horrible code that doesn't make much sense. It's actually a cut-down example from a "how not to do it" part of a course on concurrency. Nevertheless, it shouldn't be deadlocking.)

----------
components: Interpreter Core
files: hang_backtrace.txt
messages: 260273
nosy: mark.dickinson
priority: normal
severity: normal
status: open
title: Deadlock in thread.join
type: behavior
versions: Python 2.7
Added file: http://bugs.python.org/file41920/hang_backtrace.txt

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue26360>
_______________________________________