[Idle-dev] Idle Testing: Recent Thoughts, mock_tk

[Terry Reedy]

With other people working on Idle tests, I want to share my recent 
experience. (Some of this should go in idle_test.README.txt or another 
doc, when I have time.)

Idle tests are written with the unittest module, to be runnable with 
unittest.main. This mean no test_main() function (part of the older 
regrtest system), which is being removed from older test modules. (Over 
10 were changed in just the last day.)

Idle functions and class methods mostly focus on either creating or 
operating  gui or on non-gui calculations. This suggests gui tests for 
the first and non-gui for the latter. The problem is that non-gui 
methods can use gui-elements, which make the test a gui-using test 
unless they are replaced with mocks.

For instance, configSectionNameDialog.GetCfgSectionNameDialog.name_ok 
(altered spelling) checks user input and returns the stripped version. 
It gets the input from a StringVar and displays error messages with 
tkMessageBox.showerror. A messagebox is obvious a gui element, but so 
are Variable subclass instances, because they require a Tk master 
object. (Since Variables are not visible widgets, this surprised me).

Module mock_tk, used in test_name_dialog, has replacements for Variables 
and messageboxes. See the patch for issue 18130 or the new files in 
idle_test. Since tkMessageBox is a configSectionNameDialog global name 
(bound to a module), it is replaced in the module itself (for the 
duration of the test) by a mock messagebox, whose instances simply 
record the inputs for later examination. (This is colloquially called 
monkeypatching.) The fact that module functions and class methods can be 
accessed and called the same way means the module can be replaced by a 
class.

The StringVar accessed by name_ok is an instance attribute. So it is 
replaced by its mock in the dummy class defined in the test module. The 
dummy class provides a gui-free environment with the minimum attributes 
needed for the function to operate and be tested. Python makes this all 
easier than I expected.

I have a patch that does something similar for one of the three 
calculation methods in GrepDialog.GrepDialog. I hope to apply it soon, 
along with tests for the other two.

 From 18104: Idle: make human-mediated GUI tests usable
"23 of the 62 idlelib/*.py files have an 'if __name__ ...' test that 
brings up a tkinter gui widget and waits for the human tester to do ... 
something." The work on configSectionNameDialog started with getting its 
'test' to run and then deciding on and writing minimal instructions for 
actually testing it. That lead to automation of much of the checking. 
The residual minimum for humans is to check the visual appearance. For 
instance, the buttons were not centered until I removed the 'uncenter' 
argument.

18103: Create a GUI test framework for Idle; needs more work

-- 
Terry Jan Reedy