[Python-Dev] Making runpy.run_module *not* thread-safe

[Nick Coghlan]

While it may seem like going backwards, I actually have a legitimate reason 
for getting rid of the current thread safety mechanism in runpy.run_module. It 
appears Guido's doubts about the approach I used are entirely justified.

When the alter_sys flag is set, runpy.run_module pokes around a bit in the sys 
module. Because one of the things it modifies is sys.modules (putting a 
partially initialised module there), it currently uses the import lock to make 
this tinkering thread-safe.

However, if you try using "-m test.regrtest" you'll notice that 
test_threaded_import gets skipped because the import lock is held in the main 
application script. This is *not* the case when invoking regrtest directly via 
the filesystem - in that case, the runtime simply puts the partially 
initialised __main__ module into sys.modules, and leaves it to the application 
to decide whether or not that module is safe to import.

That's all well and good for regrtest.py, but any program that actually 
spawned additional threads that attempted to import modules would likely see 
deadlocks in very short order.

Removing the locking from run_module() makes it more like the PyRun C APIs - 
it puts the extra module in sys.modules, but leaves it up to the invoking code 
to decide whether or not to grab the import lock (or some other 
synchronisation object) during execution. The import lock will still be held 
while trying to *find* the requested module and the various optimisation 
caches in sys are updated.

If I don't hear any objections, I'll switch SVN (along with PEP 338 and the 
docs patch) to the new (non-thread-safe) semantics sometime this weekend.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org