[docs] [issue31761] Failures and crashes when running tests by import.

[Terry J. Reedy]

Terry J. Reedy <tjreedy at udel.edu> added the comment:

I am re-opening because there is a bug, a conflict between doc and code comment and actual behavior, and one which has nothing to do with IDLE.  The question is whether to fix it or to delete the invitation to invoke the bug.

The reason the devguide gives an alternative for running tests is so that someone with only installed Python and no command line could still test their installation and even contribute a patch on the tracker.

On 2000-08-23, Tim Peters add the following comment to test.autotest:
# This should be equivalent to running regrtest.py from the cmdline.
# It can be especially handy if you're in an interactive shell, e.g.,
# from test import autotest.

The autotest code was later copied into test.__main__, and both are now:
  from test.libregrtest import main
  main()
So 'from test import __main__' is the same as importing autotest, and should be the same as importing and running libregrtest

To test the equivalence in the setting where either import might be useful, I opened both python and IDLE from the 3.7.0a2 Windows start icons and imported autotest.  The test deterministicly crashes after test_marshal.  I call the stoppage a crash because wrapping the import in try: ... except BaseException has no effect.  The Python console just vanishes.  IDLE restarts.
...
0:05:51 [207/407/5] test_marshal

=============================== RESTART: Shell =========================
>>>

So the claimed equivalence in worse than just not true.  There were, I believe, 3 failures in the Python console.  The extra 2 in IDLE are due to the #25588 fix guarding against sys.stderr being None is needed in a couple of other places.

In any case, either running tests by importing should be made to work (again, presuming it once did) or deprecated and autotest removed.

----------
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open
title: regrtest: faulthandler.enable() fails with io.UnsupportedOperation: fileno when run from IDLE -> Failures and crashes when running tests by import.
versions:  -Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue31761>
_______________________________________