[Python-Dev] Conventions for functional tests, PyUnit?

[Raymond Hettinger]

> First, it seems the standard place for functional tests is in
> 
> if __name__ == "__main__"
> 
> right?
> 
> Some (most?) of the standard library functional tests seem to be a
bunch
> of commands that are simply supposed not to crash (urllib2) or that
are
> little test programs that need to be invoked with the correct
arguments
> (ftplib).
> 
> Would it be a good thing to attempt to PyUnit-ize these functional
tests
> so you can run them with no command line arguments and get clear
> indications of how many tests are passing and failing?  Or, if not
PyUnit,
> just ad-hoc tests that indicate success / failure?
> 
> Should some be left intact and new functional tests added?  For
example,
> ftplib has a little test program tacked on at its end, perhaps that
should
> be retained in some form?


Ideally, all repeatable, self verifying tests should be coded using the
unittest module and placed in Lib/test.

Walter Dorwald is actively working on converting tests to unittest form.
At the same time, he is running a coverage analysis tool and adding
tests to increase coverage.  Also, he is factoring out some tests that
are common to multiple modules.  If you want to assist, I recommend
working with him directly and having him give a 100% review to any
proposed changes.  For an example of that work, see:
www.python.org/sf/736962


Some of tests in the module itself are being left intact for various
reasons.  For instance, the random module still has a few tests that
need a human eye to authenticate.  Some, like the ones in urllib2 are
normally only run when someone updates the module -- they should be left
in for convenience but could also have a counterpart in Lib/test so long
as the test were marked to run only when -u network was enabled.  As you
pointed out, some are run interactively using command line arguments --
these are a development tool and should be left in.  Also, some modules
need to be left alone because they are maintained elsewhere.


Raymond Hettinger