[Python-ideas] Location of tests for packages

[Ron Adam]

Moving these suggestions to python ideas as they are not immediately 
relevant to the current python dev discussion at this time. ;-)

I'm really just wondering what others think, is this something worth 
working on?


On 01/24/2011 01:46 PM, Raymond Hettinger wrote:
> Right now, the tests for the unittest package are under the package
> directory instead of Lib/test where we have most of the other tests.
>
> There are some other packages that do the same thing, each for their own
> reason.
>
> I think we should develop a strong preference for tests going under
> Lib/test unless there is a very compelling reason.

> * For regrtest to work, there still needs to be some file in Lib/test
> that dispatches to the alternate test directory.

Currently tests are mostly separate from the modules.  Mostly separate 
because, some modules have doctests in them, and/or a test() function to 
run tests.  But the test function name isn't special in any way as far as I 
know.  It's usually _test(), but it could be something else.

Would it help things to have a special __test__ name?  ie... special to 
python, in that module.__test__() can be depended on to run the tests for 
that module? (or package)


I've found it useful to add a -T option to my own python applications to 
run tests.  But moving it from being a module option to a python option, 
would make that even nicer. Where python -T modulename called the 
__test__() function. (or evoke the tests in another way.)

With a dependable way to evoke the tests no matter where they are located, 
it then becomes just a matter of iterating the list of modules in the 
library (or a directory) to run all the tests.


But then, what's the  best way to actually do that?


The current method uses pattern matching ... (not my first choice for sure)

python -m unittest discover -s project_directory -p '*_test.py'
python -m unittest discover project_directory '*_test.py'

Those lines are way too long in my opinion.


In the above, the test functions need to begin with an underscore.  I'm not 
sure the discoverable test function should be private,  The very act of 
finding them suggests they should be a public, but special API, rather than 
a private API to me.  If they were only accessed from inside the module or 
package they are in, then being private makes sense.


One choice is that __test__ is special to doctest and unittest only.

python -m unittest module    # runs module.__test__() if it exists.


Or a little more far out ...

The __test__() function could be called with "python -T module".  But then 
it seems that maybe we could also have a __main__() function be called with 
"python -M module".  (?)


Alternately ...  "python -T module" could alter the name of the module.

if __name__ == "__main__":
    main()

elif __name__ == "__test__":
    _test()                     # a private name is ok here.


Cheers,

Ron