[issue3088] test_multiprocessing hangs on OS X 10.5.3

Amaury Forgeot d'Arc report at bugs.python.org
Sun Jun 29 01:41:28 CEST 2008


Amaury Forgeot d'Arc <amauryfa at gmail.com> added the comment:

After 4 hours spent with the debugger I think I have an explanation and
a correction of one of the problems.
Note that this only affects the "Manager" tests.

- The threading.local type overrides getattr and setattr: on every call,
it fetches a dictionary stored in the current ThreadState, and installs
it in its self->dict member, allowing the regular
PyObject_Generic[Get|Set]Attr functions to actually work on thread-local
data.
- Of course, for this to work the GIL must not be released in any way
between the moment self->dict is set, until object's dictionary is
retrieved inside PyObject_Generic[Get|Set]Attr. This condition makes the
code a bit fragile.
With current 2.6b1, there is no path in PyObject_Generic[Get|Set]Attr
that releases the GIL before the dict is retrieved - no decref, no jump
into python code (at least when the attribute name is a string). It's
almost a miracle, indeed.
- The problem is in threadmodule.c, the paragraph starting at line 285:
the first time the local object is accessed, it calls the __init__
method of the threading.local derived class. And this calls python code,
and can release the GIL.
- The consequences are dramatic: this let the possibility for another
thread to start, use the same local object, install its thread-local
dict in self->dict, and stop here; the first thread will wake up with
the wrong dict...
- in the multiprocessing case, a remote call to a synchronization
function grabs the wrong channel, and get bogus results... hence the
"cannot wait on un-aquired (sic) lock" message.

The actual correction is very short: only one "else" to remove... Here
is a patch with a test.

Added file: http://bugs.python.org/file10766/threadlocal.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue3088>
_______________________________________


More information about the Python-bugs-list mailing list