[Python-bugs-list] [ python-Bugs-683658 ] PyErr_Warn may cause import deadlock

[SourceForge.net]

Bugs item #683658, was opened at 2003-02-10 11:26
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470

Category: Python Interpreter Core
Group: Python 2.3
Status: Open
Resolution: None
Priority: 6
Submitted By: Mark Hammond (mhammond)
Assigned to: Nobody/Anonymous (nobody)
Summary: PyErr_Warn may cause import deadlock

Initial Comment:
PyErr_Warn() does an implicit import.  Thus, if
PyErr_Warn() is called on a thread while the main
thread holds the import lock, and the main thread then
subsequently waits for the child thread, Python will
deadlock.  The builtin 'apply' now calls PyErr_Warn(),
so simply calling 'apply' on another thread may cause
the deadlock.

Attaching a sample case.  Executing 'python -c "import
hang_apply"' will cause Python to deadlock.  Commenting
out the call to "apply" in that file will prevent the
deadlock (even though the apply() in question is
effectively a no-op)

The example is a little contrived, but is extracted
from real code of mine that saw this hang.  The code
path is:

* Main thread acquires import lock to import the module.
* Module import spawns a new thread.  This new thread
calls apply()
* builtin_apply() calls PyErr_Warn
* PyErr_Warn does an import of the warnings module, and
attempts to acquire the import lock.
* Main thread still waiting for child thread to
complete, but still holds import lock.

Note that removing the call to PyErr_Warn() in
builtin_apply also solves this particular hang -
however, it seems like this is a potential time bomb. 
A potential solution would be to prevent PyErr_Warn
from doing an import - this would mean importing
'warnings' at startup, and keeping a static reference
in errors.c.  Other thoughts welcome.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=683658&group_id=5470