[issue20484] calling pydoc.Helper()('modules') in the test suite sometimes causes failures

[Eric Snow]

Eric Snow added the comment:

TL;DR new tests (improving coverage) uncovered existing "bugs".  We should probably disable the tests for now.

I'm glad you found this.  Out of curiosity, how often do you run the test suite against a clean checkout?  Typically I only run it out of the same "tainted" directory that I develop in and not very frequently with -w.  Have you seen related failures on any buildbots?

Anyway, it looks like the actual pydoc changes in the patch aren't at fault.  Rather, the 3 test_modules* tests I added are.  Specifically, something in pydoc.Helper (i.e. help() in the REPL) is the problem.  That's what I get for trying to add test coverage (where there was none) for code I'm fixing! <wink>

If I recall correctly, passing 'modules' to help() does something funny like actually loading *every* module it can find.  I can understand how this might have side effects, maybe expose bugs in other modules, and even cause weird failures when running the test suite! :P

I'm not sure what it will take to get this sorted out.  This may actually be the way that pydoc.Helper()('modules') is supposed to work.  In that case we'd need to fix the modules that are having an issue, namely disutils (and probably logging).  We should open separate issues for each module that needs fixing.  Until then we should probably disable those three tests, particularly for the upcoming 3.4 release (and rc2 if we can squeeze it in).

Any objections before I disable those 3 tests?

----------
nosy: +eric.araujo, vinay.sajip
title: test_pydoc can alter execution environment causing subsequent test failures -> calling pydoc.Helper()('modules') in the test suite sometimes causes failures
type:  -> behavior

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20484>
_______________________________________