[Python-Dev] Making runpy.run_module *not* thread-safe
ncoghlan at iinet.net.au
Fri Mar 17 18:53:52 CET 2006
While it may seem like going backwards, I actually have a legitimate reason
for getting rid of the current thread safety mechanism in runpy.run_module. It
appears Guido's doubts about the approach I used are entirely justified.
When the alter_sys flag is set, runpy.run_module pokes around a bit in the sys
module. Because one of the things it modifies is sys.modules (putting a
partially initialised module there), it currently uses the import lock to make
this tinkering thread-safe.
However, if you try using "-m test.regrtest" you'll notice that
test_threaded_import gets skipped because the import lock is held in the main
application script. This is *not* the case when invoking regrtest directly via
the filesystem - in that case, the runtime simply puts the partially
initialised __main__ module into sys.modules, and leaves it to the application
to decide whether or not that module is safe to import.
That's all well and good for regrtest.py, but any program that actually
spawned additional threads that attempted to import modules would likely see
deadlocks in very short order.
Removing the locking from run_module() makes it more like the PyRun C APIs -
it puts the extra module in sys.modules, but leaves it up to the invoking code
to decide whether or not to grab the import lock (or some other
synchronisation object) during execution. The import lock will still be held
while trying to *find* the requested module and the various optimisation
caches in sys are updated.
If I don't hear any objections, I'll switch SVN (along with PEP 338 and the
docs patch) to the new (non-thread-safe) semantics sometime this weekend.
Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia
More information about the Python-Dev