[Python-Dev] regression tests with regrtest.py -r

[Tim Peters]

Here's one cause of current -r failures on Windows:

C:\Code\python\PCbuild>python ../lib/test/regrtest.py \
                              test_pty test___all__
test_pty
test test_pty skipped -- No module named termios
test___all__
test test___all__ failed -- pty has no __all__ attribute
1 test failed:
    test___all__
1 test skipped:
    test_pty
Those skips are all expected on win32.


It's "the usual":  importing pty fails on Windows, but test_pty then leaves
behind a bogus pty module object in sys.modules, which fools test___all__.
Note that pty.py already has this atrocity from a previous round of -r
wrestling:

"""
# Absurd:  import termios and then delete it.  This is to force an attempt
# to import pty to raise an ImportError on platforms that lack termios.
# Without this explicit import of termios here, some other module may
# import tty first, which in turn imports termios and dies with an
# ImportError then.  But since tty *does* exist across platforms, that
# leaves a damaged module object for tty in sys.modules, and the import
# of tty here then appears to work despite that the tty imported is junk.
import termios
del termios

import tty
"""

test___all__ also tries to compensate for pty (among others) insanity:

"""
def check_all(modname):
    names = {}
    try:
        exec "import %s" % modname in names
    except ImportError:
        # Silent fail here seems the best route since some modules
        # may not be available in all environments.
        # Since an ImportError may leave a partial module object in
        # sys.modules, get rid of that first.  Here's what happens if
        # you don't:  importing pty fails on Windows because pty tries to
        # import FCNTL, which doesn't exist.  That raises an ImportError,
        # caught here.  It also leaves a partial pty module in sys.modules.
        # So when test_pty is called later, the import of pty succeeds,
        # but shouldn't.  As a result, test_pty crashes with an
        # AttributeError instead of an ImportError, and regrtest interprets
        # the latter as a test failure (ImportError is treated as "test
        # skipped" -- which is what test_pty should say on Windows).
        try:
            del sys.modules[modname]
        except KeyError:
            pass
        return
"""

I've added another hack to regrtest.py to worm around the test_pty followed
by test___all__ case (more generally, any test_xxx followed by test___all__
case, when test_xxx fails due to leaving behind a damaged module object for
xxx).  However, that still doesn't fix

    test_rlcompleter

followed by

    test___all__

on Windows.  In that case, test_rlcompleter leaves behind a damaged
rlcompleter module, because rlcompleter.py does exist on Windows (it fails
due to the lack of the readline module).

leaving-behind-incomplete-module-objects-still-seems-like-a-bad-idea-
    to-me-ly y'rs  - tim