On 5 March 2013 05:34, Brett Cannon
On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw
wrote: On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
It is of course possible for subunit and related tools to run their own implementation, but it seems ideal to me to have a common API which regular unittest, nose, py.test and others can all agree on and use : better reuse for pretty printers, GUI displays and the like depend on some common API.
And One True Way of invoking and/or discovering how to invoke, a package's test suite.
How does unittest's test discovery not solve that?
Three reasons
a) There are some bugs (all filed I think) - I intend to hack on
these in the near future - that prevent discovery working at all for
some use cases.
b) discovery requires magic parameters that are project specific
(e.g. is it 'discover .' or 'discover . lib' to run it). This is
arguably a setup.py/packaging entrypoint issue.
c) Test suites written for e.g. Twisted, or nose, or other
non-stdunit-runner-compatible test runners will fail to execute even
when discovered correctly.
There are ways to solve this without addressing a/b/c - just defining
a standard command to run that signals success/failure with it's exit
code. Packages can export a particular flavour of that in their
setup.py if they have exceptional needs, and do nothing in the common
case. That doesn't solve 'how to introspect a package test suite' but
for distro packagers - and large scale CI integration - that doesn't
matter.
For instance testrepository offers a setuptools extension to let it be
used trivially, I believe nose does something similar.
Having something that would let *any* test suite spit out folk's
favourite test protocol de jour would be brilliant of course :).
[junit-xml, subunit, TAP, ...]
-Rob
--
Robert Collins