[Python-Dev] Whole bunch of test failures on OSX

[Tim Peters]

[Neal Norwitz, on -R testing]
> ...
> For the latest results, see:
> http://docs.python.org/dev/results/make-test-refleak.out
>
> Several tests fail consistently with -R.  These are the most recent
> from the link above:  test_decimal test_difflib test_logging
> test_optparse test_warnings.
>
> It would be great if someone would figure out why these tests fail
> when running under -R and fix them.

Like anyone wants to spend their life guessing what reload() actually
does <0.6 wink>.

C:\Code\python\PCbuild>python
Python 2.5a0 (trunk:43548M, Apr  1 2006, 21:44:15) [MSC v.1310 32 bit
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from test import test_warnings
>>> test_warnings.test_main()  # first time it's fine
test_filtering (test.test_warnings.TestModule) ... ok
test_warn_default_category (test.test_warnings.TestModule) ... ok
test_warn_specific_category (test.test_warnings.TestModule) ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.031s

OK
>>> reload(test_warnings)  # then the dreaded reload(), and it fails
<module 'test.test_warnings' from 'C:\Code\python\lib\test\test_warnings.pyc'>
>>> test_warnings.test_main()
test_filtering (test.test_warnings.TestModule) ... ERROR
test_warn_default_category (test.test_warnings.TestModule) ... ERROR
test_warn_specific_category (test.test_warnings.TestModule) ... ERROR

======================================================================
ERROR: test_filtering (test.test_warnings.TestModule)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_warnings.py", line 68, in test_filtering
    self.assertEqual(msg.message, text)
AttributeError: WarningMessage instance has no attribute 'message'

======================================================================
ERROR: test_warn_default_category (test.test_warnings.TestModule)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_warnings.py", line 42, in
test_warn_default_category
    self.assertEqual(msg.message, text)
AttributeError: WarningMessage instance has no attribute 'message'

======================================================================
ERROR: test_warn_specific_category (test.test_warnings.TestModule)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Code\python\lib\test\test_warnings.py", line 57, in
test_warn_specific_category
    self.assertEqual(msg.message, text)
AttributeError: WarningMessage instance has no attribute 'message'

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (errors=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Code\python\lib\test\test_warnings.py", line 85, in test_main
    test_support.run_unittest(TestModule)
  File "C:\Code\python\lib\test\test_support.py", line 300, in run_unittest
    run_suite(suite, testclass)
  File "C:\Code\python\lib\test\test_support.py", line 284, in run_suite
    raise TestFailed(msg)
test.test_support.TestFailed: errors occurred in test.test_warnings.TestModule
>>>

Figuring out why that happens is pretty much a nightmare.  "Fixing it"
requires changes to both regrtest and test_warnings:

"""
Index: Lib/test/regrtest.py
===================================================================
--- Lib/test/regrtest.py        (revision 43548)
+++ Lib/test/regrtest.py        (working copy)
@@ -536,12 +536,10 @@
                     sys.path_importer_cache.update(pic)
                     dircache.reset()
                     linecache.clearcache()
-                if indirect_test:
-                    def run_the_test():
-                        indirect_test()
-                else:
-                    def run_the_test():
-                        reload(the_module)
+                def run_the_test():
+                    reload(the_module)
+                    if indirect_test:
+                        getattr(the_module, "test_main")()
                 deltas = []
                 repcount = huntrleaks[0] + huntrleaks[1]
                 print >> sys.stderr, "beginning", repcount, "repetitions"
Index: Lib/test/test_warnings.py
===================================================================
--- Lib/test/test_warnings.py   (revision 43548)
+++ Lib/test/test_warnings.py   (working copy)
@@ -84,5 +84,9 @@
 def test_main(verbose=None):
     test_support.run_unittest(TestModule)

+# Obscure hack so that this test passes after reloads (regrtest -R).
+if '__warningregistry__' in globals():
+    del globals()['__warningregistry__']
+
 if __name__ == "__main__":
     test_main(verbose=True)
"""

The change to regrtest may fix other -R cases (for example, it appears
to repair test_decimal), but may introduce new failures too.  If you
try it and find it's a net win ;-), feel free to check it in.