[issue18104] Idle: make human-mediated GUI tests usable

[Terry J. Reedy]

Terry J. Reedy added the comment:

Some follow-up after revising the test for GetCfgSectionNameDialog in  configSectionNameDialog.py (#18130):

1. I am no longer concerned about automated discovery. Anyone adding a widget can just as easily add something to a master list as add a file to be discovered. I meant to type up and post my handwritten list of files with widget tests, with the status of each. I will try to find it or recreate it.

2. I am not so much concerned about an automated summary. Once the tests work correctly, errors due to editing the underlying code should be caught by immediate testing.  The model I used is

if __name__...:
  <run unittests, as usual>
  <if file defines visible widget, also run human test>

where <run human test> currently means to open a tk window with a button to start the test, but which can instead just be closed [x] to skip the test. My idea is that after editing, one should always run the automated test (if there is one, which there eventually will be) and maybe run the visual test.

3. For #18130, I left the test code in the 'if name' clause. To run all such tests from a separate master htest file, it needs to be moved into a function. I am thinking of '_htest'. (Some files already have the visual test code pulling into a 'main' function, but to me that is too similar to 'unitest.main' and similar.) Or perhaps the code should be moved to an idle_test/htest.py file, but with a means to run just the one test from the module file.

4. For this test, the widget has a message parameter, which I used for test hints. Other widgets do not have such, and the test hints will have to be in the driver window.

I need to pick, say, 3 tests and do some experiments.

----------
nosy: +sahutd

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18104>
_______________________________________