[Python-ideas] Adding a test discovery into Python

[Guilherme Polo]

On Sun, Feb 1, 2009 at 6:46 PM, Brett Cannon <brett at python.org> wrote:
> On Sun, Feb 1, 2009 at 09:10, Guilherme Polo <ggpolo at gmail.com> wrote:
>> Hi,
>>
>> I believe it would be good to include a test discovery into Python,
>> right now I notice all the following packages:
>>
>> bsdbb, ctypes, distutils, email, json, lib2to3 and lib-tk (tkinter in py3k)
>>
>
> ... and importlib.

My bad, I was even looking at it other day but when I compiled this
list of packages I looked only in trunk and forgot about the py3k
branch.

>> So.. is there any chance we can enter in agreement what features would
>> be useful in a test discovery that could be included with Python ? I
>> for myself do not have fancy wishes for this one, I would be happy
>> with something that would collect unittests, inside packages and
>> subpackages, with some fixed patterns and let me run them with
>> test.test_support.run_unittests, or maybe something that would collect
>> unittests and doctests and have something like run_tests in
>> test.test_support. But then I believe this wouldn't be good enough to
>> substitute any of the current tools, making the addition mostly
>> useless.
>
> Yep.

To the agreement question ?

> I want to be able to state "find the tests in this package and
> search the entire package top-down looking for tests to run". The only
> trick is if you store the tests in something other than on the file
> system directly (e.g. zipfile). That would require a little bit more
> work like having modules listed in the __all__ value of the package
> and use that to know which modules to look in.

This made me remember about another two features I think would be
useful to have.

One of these features would allow me to restrict from which packages I
want to collect tests, this is useful even in "flat packages" (no
sub-packages) like tkinter where some of its modules possibly depend
on external packages in order to be tested/executed. This adds the
possibility to skip tests of single modules instead of entire
packages.

Other one would be able to check the requirements of a test and decide
to collect it or not based on some parameters given. So I could
collect all the tests that require a GUI, and others that do not, for
example. This adds the possibility to skip single tests, or at least
the possibility to divide the tests before running.

> -Brett
>


-- 
-- Guilherme H. Polo Goncalves