Re: [Twisted-Python] Re: Using twisted.trial with nose unittest system
On 03:32 pm, antoine.pitrou@wengo.com wrote:
Hi Jean-Paul,
The one practical problem which this method of integration has is that it will not allow unit tests to run child processes. So, one wouldn't be able to run Twisted's test suite with nose using this. Of course, if one has no interest in running tests which run child processes, then this may not be a big deal.
Do you mean because the reactor does not receive signals?
Yes. Also, it's highly likely that the tests won't stop cleanly on ^C. More serious than the process-running issue, from my point of view, is the fact that if you forget the "@deferred" decorator on a test that uses any reactor API, or code that internally _uses_ any reactor API, you will put your tests into a non-deterministic state, and although they'll pass _most_ of the time, they will die depending on the load characteristics of your machine at the time and (insert long-winded rant about the evil things threads do to your code here). Tests are exactly the place you never want this to happen :). The documentation for this module should be clear on the fact that Twisted-using tests are _absolutely required_ to use the decorator, even if they don't need to wait on a deferred. Of course, that implies that the "you must return a twisted Deferred" message should probably be removed and that case dealt with. Also, setUp and tearDown methods also need the decorator applied, I think, not just test methods. All in all it seems like it would be simpler to just use trials' TestCase class with nose. Is there any reason not to?
Let me give a few points as for nose vs. Trial :
You are welcome to use nose, of course, but some of these reasons sounded kind of flimsy to me.
- nose does not depend on Twisted. Unless all your Python projects make use of Twisted, you certainly don't like having to use a different testing tool depending on the project.
Trial doesn't force your code to make use of Twisted, and as Mr. Lange has declared previously, it goes to some lengths to be compatible with stdlib unit tests. It can run and discover stdlib tests, as well as being run by the stdlib runner, and I've used it to test tiny projects which do not use Twisted code anywhere. So, Trial only requires Twisted insofar as Nose requires Nose. It's part of its implementation. If you use Twisted for _any_ of your development, it's already installed.
- nose has nice features, including the ability to write tests as functions without having to write classes
This just sounds like TIMTOWTDI to me. I *like* the fact that all test methods look the same ;).
- nose has a plugin system which already provides several extensions
In the spirit of completeness, see trial's --help-reporters and --reporter option. These are discovered using plugins. Please do not take this argument seriously though. :) There is widespread consensus that this plugin API is not very good and it is likely to change. Still, it exists and you can use it. See: http://twistedmatrix.com/trac/browser/sandbox/exarkun/merit/trunk/twisted/pl... Although, perhaps the utility of the plugin system is decreased more by the fact that nobody knows about it than by its sub-optimal API, which leads into your next point...
- nose is documented
This is a very, very good point. Trial's documentation is extremely weak. Does anyone (and by "anyone" I mean "anyone but JML, who has already done more work on trial than any reasonable person ever would") want to volunteer to write a "test driven development with Twisted" lore document? I've introduced many people to Twisted in recent weeks, and each time I ended up having to do a live demonstration of how to write a test module, how to run the trial command line, and (for those lucky enough to use emacs) how to get twisted-dev.el loaded so that f9 will do the right thing. This stuff isn't clear even to people already familiar with Python and unit testing, let alone to people who are learning TDD for the first time with trial.
On Wed, 07 Feb 2007 16:35:10 -0000, glyph@divmod.com wrote:
On 03:32 pm, antoine.pitrou@wengo.com wrote:
Hi Jean-Paul,
The one practical problem which this method of integration has is that it will not allow unit tests to run child processes. So, one wouldn't be able to run Twisted's test suite with nose using this. Of course, if one has no interest in running tests which run child processes, then this may not be a big deal.
Do you mean because the reactor does not receive signals?
Yes. Also, it's highly likely that the tests won't stop cleanly on ^C.
Haa haha haa hahaha haaa haa haa haa heee haaa ha ha ha. Jean-Paul
Hi, Le mercredi 07 février 2007 à 16:35 +0000, glyph@divmod.com a écrit :
Yes. Also, it's highly likely that the tests won't stop cleanly on ^C.
Depends on what you mean by "cleanly". I haven't noticed any annoying behaviour when killing tests in the middle. Regardless of ^C, there is a sporadic exception at shutdown due to the way Python destroys modules (it first sets the module members to None). I suppose it can happen in any situation in which Twisted is run in a separate thread: << Exception in thread Thread-1 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "threading.py", line 460, in __bootstrap File "threading.py", line 440, in run File "/home/antoine/nose-svn/nose/twistedtools.py", line 52, in <lambda> File "/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py", line 218, in run File "/usr/lib/python2.5/site-packages/twisted/internet/posixbase.py", line 231, in mainLoop <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'msg' Unhandled exception in thread started by Error in sys.excepthook: Original exception was:
More serious than the process-running issue, from my point of view, is the fact that if you forget the "@deferred" decorator on a test that uses any reactor API, or code that internally _uses_ any reactor API, you will put your tests into a non-deterministic state, and although they'll pass _most_ of the time, they will die depending on the load characteristics of your machine at the time and (insert long-winded rant about the evil things threads do to your code here). Tests are exactly the place you never want this to happen :).
That's certainly right, although you should get messages about uncaught exceptions or unhandled errbacks, so you'll know that something went wrong. What happens with Trial when you forget to return the Deferred the test should wait on?
Also, setUp and tearDown methods also need the decorator applied, I think, not just test methods.
Setup methods at least, if they do something asynchronous.
All in all it seems like it would be simpler to just use trials' TestCase class with nose. Is there any reason not to?
Given that Trial is a self-contained tool, and given that nose also has its own test running and collecting, I simply didn't expect this to work or to keep working accross versions of both tools.
Trial doesn't force your code to make use of Twisted, and as Mr. Lange has declared previously, it goes to some lengths to be compatible with stdlib unit tests.
Of course, but to use Trial you must install Twisted don't you ? Which is quite a big dependency, and Twisted does not yet support being easy_install'ed. Regards Antoine.
On 2/8/07, glyph@divmod.com <glyph@divmod.com> wrote:
On 03:32 pm, antoine.pitrou@wengo.com wrote:
- nose is documented
This is a very, very good point. Trial's documentation is extremely weak.
Does anyone (and by "anyone" I mean "anyone but JML, who has already done more work on trial than any reasonable person ever would") want to volunteer to write a "test driven development with Twisted" lore document?
It's on my todo list. ;) cheers, jml
participants (4)
-
Antoine Pitrou -
glyph@divmod.com -
Jean-Paul Calderone -
Jonathan Lange