[Idle-dev] Non-buildbot human mediated tests

Saimadhav Heblikar saimadhavheblikar at gmail.com
Sun Apr 27 07:34:57 CEST 2014

In this email, I'd like to detail how i plan to move forward on making
human mediated non-buildbot tests in IDLE.

Existing discussions - http://bugs.python.org/issue18104

A file called htest.py to be present in idlelib/idle_test. This file
will have a root gui. This gui will contain information about the
module. It will contain information useful to the tester - what
aspects to test, what are the results to be expected etc. Formatting
this is trivial. It will have a button to start the test(run the
module in hand).

Once the testing is over, we see if any exception was raised. If yes,
print the same to console and fail the test. We can also ask the
tester to choose one of {Pass,Error etc}, which will also be printed
to console.

Moving to the design aspect,

1. How to pass information to htest?
By information, i am referring to "help information string" for
tester, arguments to be passed to htest etc. This information can be
stored in a dict.
There are two possibilities -
 a) store the information related to each module in the module itself.
The advantages are obvious to see  - Its better organized and will
also give the developer an idea about the module
b) store all the information per module in htest.py . I don't feel
this option would help, but mentioning here to see if there are any
advantages of going this way.

2. Should I consider the possibility of ALL these tests being run together?
(Much the same way as unitttests are run now - like "python -m
unittest -v idlelib.idle_test").

In this case, i plan to keep it simple. Just display the dialog's as
if they are being run individually,(one after another), and print the
result to console.
This can done using a simple loop.

Have i missed any other aspect?

I am awaiting your feedback.

Saimadhav Heblikar
GSoC 2014 Student
Overview : http://blog.saimadhav.com/2014/04/gsoc-2014-workflow-resources.html

NS: I have made a rough overview of which modules need gui testing,
whether they are indirectly being tested etc. at
http://bit.ly/1k5JmHn. Feel free to comment.

More information about the IDLE-dev mailing list