On 4 Mar 2013, at 20:00, Robert Collins email@example.com wrote:
On 5 March 2013 05:34, Brett Cannon firstname.lastname@example.org wrote:
On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw email@example.com wrote:
On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
It is of course possible for subunit and related tools to run their own implementation, but it seems ideal to me to have a common API which regular unittest, nose, py.test and others can all agree on and use : better reuse for pretty printers, GUI displays and the like depend on some common API.
And One True Way of invoking and/or discovering how to invoke, a package's test suite.
How does unittest's test discovery not solve that?
Three reasons a) There are some bugs (all filed I think) - I intend to hack on these in the near future - that prevent discovery working at all for some use cases.
The only discovery related issues I'm aware of are:
* Issue 16079 (filed by you) - trivial to fix just needs a test * Issue 15010 obscure and unlikely to be an issue for standard discovery
I'm not aware of any other discovery related issues. Please let me know (or add me as nosy) to them.
b) discovery requires magic parameters that are project specific (e.g. is it 'discover .' or 'discover . lib' to run it). This is arguably a setup.py/packaging entrypoint issue.
This was addressed by Barry - and yes discovery has to be done with the right parameters. If you layout your project in a particular way then "python -m unittest discover" in the project root will just work. This is project specific metadata though and not a particular problem of any testing library.
c) Test suites written for e.g. Twisted, or nose, or other non-stdunit-runner-compatible test runners will fail to execute even when discovered correctly.
There are ways to solve this without addressing a/b/c - just defining a standard command to run that signals success/failure with it's exit code. Packages can export a particular flavour of that in their setup.py if they have exceptional needs, and do nothing in the common case. That doesn't solve 'how to introspect a package test suite' but for distro packagers - and large scale CI integration - that doesn't matter.
For instance testrepository offers a setuptools extension to let it be used trivially, I believe nose does something similar.
unittest2 also has setuptools compatible test command.
Having something that would let *any* test suite spit out folk's favourite test protocol de jour would be brilliant of course :). [junit-xml, subunit, TAP, ...]
-- Robert Collins firstname.lastname@example.org Distinguished Technologist HP Cloud Services _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html