Where to place imports

Christian Heimes lists at cheimes.de
Fri Jan 23 16:49:34 EST 2009


Jean-Paul Calderone schrieb:
>> BTW, if you ever find you are starting to write multi-threaded
>> applications
>> then you'll really regret it if you reuse code which does imports from
>> inside functions. If two or more threads try to import a module
>> simultaneously then one of them might find it that not everything is
>> defined in the module exists when it tries to use it.
> 
> What makes you say this?  There is an import lock (beyond the GIL) which
> should prevent a thread from ever seeing a partially initialized module.

Partially initialized modules aren't the issue here. One can easily
create a dead lock situation with a mix of threads and imports.

Example:

* module 'mod_a' is imported [import lock is acquired]
* module 'mod_a' spawns a new thread
* the new thread tries to import another module 'mod_b'

Python dead locks here and there is no chance to resolve the lock. The
import of 'mod_b' is waiting for the first import. The first import
hasn't finished because it's still executing the body of 'mod_a'.

In order to avoid the situation you *must* never import a module in a
thread other than the main thread. You should never start a thread in
the body of a submodule, too. A large application should import all its
modules first and then create its threads.

If you really need to import a module from a sub thread or a method that
might be called from a sub thread then you should use the C function
PyImport_ImportModuleNoBlock() or use sys.modules and imp.lock_held().

Christian




More information about the Python-list mailing list