On Fri, Aug 12, 2005 at 14:35 -0500, Ian Bicking wrote:
CC'ing to bring together opposing viewpoints... (this is referring to setuptools' "setup.py test" command, which expects to have a TestSuite to run)
Phillip J. Eby wrote:
Also, the 'test' command isn't "limited" to unittest, it just expects a unittest-compatible wrapper. For example, doctest has functions that wrap doctests as unittest suites, so it's quite possible to use that. As far as I'm concerned, unittest suites are the WSGI of testing; if somebody creates a fancy new test framework, they should darn well make it possible to put a unittest suite wrapper around it, so that people can integrate their existing tests. :)
Well, I don't know what exactly my opinion is. At a py.test user I don't have TestSuites for my tests. I've argued py.test should load TestCase-based tests by default, but this is kind of the opposite. I don't think it would be that hard to produce such suites; the test items that py.test collects could just be stuffed into a TestCase.
I agree without having looked deeply into it. Providing TestSuite instances that adapt running tests in a simple way with part of the py.test machinery should be quite feasible. Would probably mean that some of py.test's execution features would be switched off (like stdout/stderr capturing). However, as Ian hints at, py.test itself is already a project independent entry point for running tests in a given directory or for a given project.
The result would be acceptable as a sort of "this package thinks it runs okay" test. It's not the frontend I'd like to give to users.
unsurprisingly, i agree :-)
That said, if "python setup.py test ..." was completely equivalent to "py.test ..." then that would be great, because though the interface would be different from projects that use unittest, the entry point would be the same (assuming py.test is installed).
would make sense, i think.
I suppose a package could add an entry point that overrides the normal setuptools test command...?
Yes, i guess something needs to be configurable there as far as i understand the situation. I also presume that "setup.py test" would allow a custom test method to actually perform the execution of tests, not just provide a TestSuite. If so, it should at best become a simple matter of how a package can signal to setuptools that it wants its tests to be handled by py.test in which case the work would be defered to the (neccessarily) installed py library when "setup.py test" is invoked. It shouldn't be required from each application to provide this glue code as it should be generic. actually, how does "setup.py test" work today and which applications/projects are integrated with it if i may ask? cheers, holger