On 5 March 2013 05:34, Brett Cannon email@example.com wrote:
On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw firstname.lastname@example.org wrote:
On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
It is of course possible for subunit and related tools to run their own implementation, but it seems ideal to me to have a common API which regular unittest, nose, py.test and others can all agree on and use : better reuse for pretty printers, GUI displays and the like depend on some common API.
And One True Way of invoking and/or discovering how to invoke, a package's test suite.
How does unittest's test discovery not solve that?
Three reasons a) There are some bugs (all filed I think) - I intend to hack on these in the near future - that prevent discovery working at all for some use cases. b) discovery requires magic parameters that are project specific (e.g. is it 'discover .' or 'discover . lib' to run it). This is arguably a setup.py/packaging entrypoint issue. c) Test suites written for e.g. Twisted, or nose, or other non-stdunit-runner-compatible test runners will fail to execute even when discovered correctly.
There are ways to solve this without addressing a/b/c - just defining a standard command to run that signals success/failure with it's exit code. Packages can export a particular flavour of that in their setup.py if they have exceptional needs, and do nothing in the common case. That doesn't solve 'how to introspect a package test suite' but for distro packagers - and large scale CI integration - that doesn't matter.
For instance testrepository offers a setuptools extension to let it be used trivially, I believe nose does something similar.
Having something that would let *any* test suite spit out folk's favourite test protocol de jour would be brilliant of course :). [junit-xml, subunit, TAP, ...]