holger krekel wrote:
Well, I don't know what exactly my opinion is. At a py.test user I don't have TestSuites for my tests. I've argued py.test should load TestCase-based tests by default, but this is kind of the opposite. I don't think it would be that hard to produce such suites; the test items that py.test collects could just be stuffed into a TestCase.
I agree without having looked deeply into it. Providing TestSuite instances that adapt running tests in a simple way with part of the py.test machinery should be quite feasible. Would probably mean that some of py.test's execution features would be switched off (like stdout/stderr capturing).
However, as Ian hints at, py.test itself is already a project independent entry point for running tests in a given directory or for a given project.
The result would be acceptable as a sort of "this package thinks it runs okay" test. It's not the frontend I'd like to give to users.
unsurprisingly, i agree :-)
Actually I mistyped -- for developers (or potential developers) I think py.test's full featureset and frontend should be available. However, for mere library users there's a use case where they want to simply know that the tests pass, that the package is ok; failures are not expected. At least, when I was talking about this with some Perl people, they specifically said they like that CPAN automatically runs tests when you download a package. I think a uniform command-line interface is nice in this case; the user is just look for "ok" or "not ok", they aren't looking for a development tool. And it's possible that "setup.py test" should be that uniform interface. That doesn't exclude the possibility of a separate command like "setup.py pytest".
I suppose a package could add an entry point that overrides the normal setuptools test command...?
Yes, i guess something needs to be configurable there as far as i understand the situation. I also presume that "setup.py test" would allow a custom test method to actually perform the execution of tests, not just provide a TestSuite.
If so, it should at best become a simple matter of how a package can signal to setuptools that it wants its tests to be handled by py.test in which case the work would be defered to the (neccessarily) installed py library when "setup.py test" is invoked. It shouldn't be required from each application to provide this glue code as it should be generic.
actually, how does "setup.py test" work today and which applications/projects are integrated with it if i may ask?
AFAIK no applications are integrated. Well, maybe PEAK... this is all rather new. It's documented here: http://peak.telecommunity.com/DevCenter/setuptools#test-build-package-and-ru... However, I believe each command to setup.py is simply an "entry_point". I can't find documentation for that at the moment. But it's something like this in the setup.py file: entry_points={ 'distutils.commands': [ 'test=test_module:test_command', ], } And then that test_command object (a class) has a specific interface (based on the distutils.Command class). So potentially a py.test-using project could do: entry_points={ 'distutils.commands': [ 'test=mypkg.commands:test']} # in mypkg.commands: from py.test.disutils import test I think that would override "setup.py test" for just that package. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org