[Python-checkins] r45631 - in python/trunk: Lib/ntpath.py Lib/test/test_threaded_import.py Lib/test/threaded_import_hangers.py Misc/NEWS

[tim.peters]

Author: tim.peters
Date: Fri Apr 21 23:18:10 2006
New Revision: 45631

Added:
   python/trunk/Lib/test/threaded_import_hangers.py   (contents, props changed)
Modified:
   python/trunk/Lib/ntpath.py
   python/trunk/Lib/test/test_threaded_import.py
   python/trunk/Misc/NEWS
Log:
SF bug #1473760 TempFile can hang on Windows.

Python 2.4 changed ntpath.abspath to do an import
inside the function.  As a result, due to Python's
import lock, anything calling abspath on Windows
(directly, or indirectly like tempfile.TemporaryFile)
hung when it was called from a thread spawned as a
side effect of importing a module.

This is a depressingly frequent problem, and
deserves a more general fix.  I'm settling for
a micro-fix here because this specific one accounts
for a report of Zope Corp's ZEO hanging on Windows,
and it was an odd way to change abspath to begin
with (ntpath needs a different implementation
depending on whether we're actually running on
Windows, and the _obvious_ way to arrange for that
is not to bury a possibly-failing import _inside_
the function).

Note that if/when other micro-fixes of this kind
get made, the new Lib/test/threaded_import_hangers.py
is a convenient place to add tests for them.


Modified: python/trunk/Lib/ntpath.py
==============================================================================
--- python/trunk/Lib/ntpath.py	(original)
+++ python/trunk/Lib/ntpath.py	Fri Apr 21 23:18:10 2006
@@ -481,27 +481,28 @@
 
 
 # Return an absolute path.
-def abspath(path):
-    """Return the absolute version of a path"""
-    try:
-        from nt import _getfullpathname
-    except ImportError: # Not running on Windows - mock up something sensible.
-        global abspath
-        def _abspath(path):
-            if not isabs(path):
-                path = join(os.getcwd(), path)
-            return normpath(path)
-        abspath = _abspath
-        return _abspath(path)
-
-    if path: # Empty path must return current working directory.
-        try:
-            path = _getfullpathname(path)
-        except WindowsError:
-            pass # Bad path - return unchanged.
-    else:
-        path = os.getcwd()
-    return normpath(path)
+try:
+    from nt import _getfullpathname
+
+except ImportError: # not running on Windows - mock up something sensible
+    def abspath(path):
+        """Return the absolute version of a path."""
+        if not isabs(path):
+            path = join(os.getcwd(), path)
+        return normpath(path)
+
+else:  # use native Windows method on Windows
+    def abspath(path):
+        """Return the absolute version of a path."""
+
+        if path: # Empty path must return current working directory.
+            try:
+                path = _getfullpathname(path)
+            except WindowsError:
+                pass # Bad path - return unchanged.
+        else:
+            path = os.getcwd()
+        return normpath(path)
 
 # realpath is a no-op on systems without islink support
 realpath = abspath

Modified: python/trunk/Lib/test/test_threaded_import.py
==============================================================================
--- python/trunk/Lib/test/test_threaded_import.py	(original)
+++ python/trunk/Lib/test/test_threaded_import.py	Fri Apr 21 23:18:10 2006
@@ -6,7 +6,7 @@
 # randrange, and then Python hangs.
 
 import thread
-from test.test_support import verbose, TestSkipped
+from test.test_support import verbose, TestSkipped, TestFailed
 
 critical_section = thread.allocate_lock()
 done = thread.allocate_lock()
@@ -25,6 +25,23 @@
     if finished:
         done.release()
 
+def test_import_hangers():
+    import sys
+    if verbose:
+        print "testing import hangers ...",
+
+    from test import threaded_import_hangers
+
+    try:
+        if threaded_import_hangers.errors:
+            raise TestFailed(threaded_import_hangers.errors)
+        elif verbose:
+            print "OK."
+    finally:
+        # In case this test is run again, make sure the helper module
+        # gets loaded from scratch again.
+        del sys.modules['test.threaded_import_hangers']
+
 # Tricky:  When regrtest imports this module, the thread running regrtest
 # grabs the import lock and won't let go of it until this module returns.
 # All other threads attempting an import hang for the duration.  Since
@@ -53,5 +70,7 @@
             print "OK."
     done.release()
 
+    test_import_hangers()
+
 if __name__ == "__main__":
     test_main()

Added: python/trunk/Lib/test/threaded_import_hangers.py
==============================================================================
--- (empty file)
+++ python/trunk/Lib/test/threaded_import_hangers.py	Fri Apr 21 23:18:10 2006
@@ -0,0 +1,42 @@
+# This is a helper module for test_threaded_import.  The test imports this
+# module, and this module tries to run various Python library functions in
+# their own thread, as a side effect of being imported.  If the spawned
+# thread doesn't complete in TIMEOUT seconds, an "appeared to hang" message
+# is appended to the module-global `errors` list.  That list remains empty
+# if (and only if) all functions tested complete.
+
+TIMEOUT = 10
+
+import threading
+
+import tempfile
+import os.path
+
+errors = []
+
+# This class merely runs a function in its own thread T.  The thread importing
+# this module holds the import lock, so if the function called by T tries
+# to do its own imports it will block waiting for this module's import
+# to complete.
+class Worker(threading.Thread):
+    def __init__(self, function, args):
+        threading.Thread.__init__(self)
+        self.function = function
+        self.args = args
+
+    def run(self):
+        self.function(*self.args)
+
+for name, func, args in [
+        # Bug 147376:  TemporaryFile hung on Windows, starting in Python 2.4.
+        ("tempfile.TemporaryFile", tempfile.TemporaryFile, ()),
+
+        # The real cause for bug 147376:  ntpath.abspath() caused the hang.
+        ("os.path.abspath", os.path.abspath, ('.',)),
+        ]:
+
+    t = Worker(func, args)
+    t.start()
+    t.join(TIMEOUT)
+    if t.isAlive():
+        errors.append("%s appeared to hang" % name)

Modified: python/trunk/Misc/NEWS
==============================================================================
--- python/trunk/Misc/NEWS	(original)
+++ python/trunk/Misc/NEWS	Fri Apr 21 23:18:10 2006
@@ -86,6 +86,9 @@
 Library
 -------
 
+- Bug #1473760: ``tempfile.TemporaryFile()`` could hang on Windows, when
+  called from a thread spawned as a side effect of importing a module.
+
 - New modules: setuptools, easy_install, and pkg_resources, to support
   building, installing, and using Python eggs, respectively.