[Twisted-Python] testing my application
Hi all, I am looking for a simple way to write test cases for my project. What is the right way to be as standard as possible? I know the application works becouse I developed a client and the server speaking the same protocol and used lots of print to have output and a sniffer to see network trafic but I understand this is not the right way. I read unittest docs, I found twisted.trial and pocked around the test cases of some twisted protocol, what I was unable to understand is how to write my taste cases. Could somebody show me the way or point me to some docs? TIA Stefano -- Stefano Canepa aka sc: sc@linux.it http://www.stefanocanepa.it Three great virtues of a programmer: laziness, impatience and hubris. Le tre grandi virtù di un programmatore: pigrizia, impazienza e arroganza. (Larry Wall)
Dnia czwartek, 20 października 2005 13:16, Stefano Canepa napisał:
Hmm, I have the similar problem. I have been using python unittest to test my application components but recently I started to write classess which return deferreds and I am not sure how to test them. First I tried to use standard unittest and install callbacks which call self.fail or self.succeed - but it seems that the errors are just ignored. Then I found on this list note about deferredResult and tried using it (keeping normal unittest) but seems the application just hangs here (for the simple reason I guess, no reactor is running) Then I found about twisted.trial.unittest. But when I replaced import unittest with from twisted.trial import unittest, I found that there is no 'main' function in this module. What should I do? Go back to normal unittest but start some reactor before unittest.main() ? Does there anywhere exist any complete ***standalone*** test example (run for itself instead of being run within whole twisted test framework)?
On Thu, 20 Oct 2005 15:19:15 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
exarkun@boson:~$ cat > test_foo.py from twisted.trial import unittest from twisted.internet import defer def myTestableFunction(x): return defer.succeed(x) def myBrokenFunction(y): return defer.fail(ZeroDivisionError("Math is hard")) def MyUnitTests(unittest.TestCase): def testTestable(self): return myTestableFunction(10).addCallback(self.assertEquals, 10) def testBroken(self): return self.assertFailure(myBrokenFunction("foo"), ZeroDivisionError) exarkun@boson:~$ trial test_foo.py Running 2 tests. test_foo MyUnitTests testBroken ... [OK] testTestable ... [OK] -------------------------------------------------- Ran 2 tests in 0.033s PASSED (successes=2) exarkun@boson:~$ Hope this helps, Jp
Thanks for the info. Running tests vai 'trial test_x.foo' has unpleasant effect of not allowing debugging but simple trick works: if __name__ == "__main__": import twisted.scripts.trial import sys sys.argv = ['-m', 'my_test' ] twisted.scripts.trial.run() (BTW, what about patching run so it could get parameters and fallback to argv only when they are not given?) Currently I have the following problem: my test script hangs. Via simplification I got to the following text (requires some postgres database): from twisted.trial import unittest from twisted.trial.util import deferredResult import psycopg from twisted.enterprise import adbapi DSN = "dbname=marcink port=5433 user=user password=pwd host=localhost" class MyTestCase(unittest.TestCase): def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) # (populating some test data) deferredResult( self.dbpool.runInteraction(self._insertTestData) ) def tearDown(self): deferredResult( self.dbpool.runOperation("DROP TABLE mytable") ); def _insertTestData(self, tx): tx.execute("CREATE TABLE mytable(id INTEGER)") tx.execute("INSERT INTO mytable(id) VALUES(1)") def testSomething(self): d = self.dbpool.runQuery("SELECT id FROM mytable") d.addCallback( lambda rows: self.assertEquals(len(rows), 1) ) When I run trial -v my_test I get my_test MyTestCase testSomething ... (and the test hangs forever)
Hmm, replying to myself, it seems that when I changed to def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) return self.dbpool.runInteraction(self._insertTestData) my test seems to work. Could anyone confirm that setUp (and tearDown) are also allowed to return deferreds? I have also another problem. In fact I need to run a few deferred-returning functions within setUp. How should I join them? I can create DeferredList but will it then be checked for failures?
Continuing replying to myself, the test seems to work even too well. In fact it insist on succeeding. For instance when I changed sql command to some rubbish, the test still succeeds. For example this test is raported as succesfully executed def testSomething(self): d = self.dbpool.runQuery("RUBBISH SELECT BLAH BLAH") return d I had to miss something. But ... what?
As one more attempt I just ugraded from twisted 1.3 to twisted 2.0 (= from twisted version present in Debian testing to twisted version present in Debian unstable). And now ........ the script which in 1.3 ignored errors but worked, *hangs(, in exactly the same way in which it used to hang when I used deferredResult.
I found the solution. The following modification caused everything to work as expected def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) self.dbpool.start() # ... (I added self.dbpool.start()) Confusing part is that start method doc tells: If you are using the reactor normally, this function does *not* need to be called. So it seems trial is *not* using reactor 'normally' - whatever does it mean.
On Thu, 20 Oct 2005 18:07:19 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
Indeed. Hopefully this will be rectified at some future point, but it is difficult to fix for now, due to the way the vast majority of existing tests are written, along with the desire to retain backwards compatibility. Jp
Maybe documentation could be slightly patched instead? Some sentence suggesting that one should call 'start' method if the application seem to hang could be added to the adbapi description... I must say (and you can see on this list ;-)) that for novice twisted programmer like me the whole thing was very confusing.
On Thu, 20 Oct 2005 16:05:03 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
Just so you know, and in case anyone reading isn't familiar with it, trial has a debug mode. If you pass the -b flag, it will automatically set up pdb, plus a couple other Twisted-specific features that come in fairly handy. twistd also supports this, letting you break into pdb at any point while the app is running, and also stopping for unhandled errors. Jp
I found it, but for not-so-hardcore python programmer, pdb is not the tool of the dreams. GUI Debugger (for instance one included in WingIDE or in Komodo or in Eric or ... is far more useful ....). And to use one it is necessary to start script within the debugger control, not to start debugger from trial. And, as I started to talk about GUIs, it would be also nice to run unittests under GUI control (I love eric3-unittest - using it even when I am not using eric environment itself).
Dnia czwartek, 20 października 2005 13:16, Stefano Canepa napisał:
Hmm, I have the similar problem. I have been using python unittest to test my application components but recently I started to write classess which return deferreds and I am not sure how to test them. First I tried to use standard unittest and install callbacks which call self.fail or self.succeed - but it seems that the errors are just ignored. Then I found on this list note about deferredResult and tried using it (keeping normal unittest) but seems the application just hangs here (for the simple reason I guess, no reactor is running) Then I found about twisted.trial.unittest. But when I replaced import unittest with from twisted.trial import unittest, I found that there is no 'main' function in this module. What should I do? Go back to normal unittest but start some reactor before unittest.main() ? Does there anywhere exist any complete ***standalone*** test example (run for itself instead of being run within whole twisted test framework)?
On Thu, 20 Oct 2005 15:19:15 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
exarkun@boson:~$ cat > test_foo.py from twisted.trial import unittest from twisted.internet import defer def myTestableFunction(x): return defer.succeed(x) def myBrokenFunction(y): return defer.fail(ZeroDivisionError("Math is hard")) def MyUnitTests(unittest.TestCase): def testTestable(self): return myTestableFunction(10).addCallback(self.assertEquals, 10) def testBroken(self): return self.assertFailure(myBrokenFunction("foo"), ZeroDivisionError) exarkun@boson:~$ trial test_foo.py Running 2 tests. test_foo MyUnitTests testBroken ... [OK] testTestable ... [OK] -------------------------------------------------- Ran 2 tests in 0.033s PASSED (successes=2) exarkun@boson:~$ Hope this helps, Jp
Thanks for the info. Running tests vai 'trial test_x.foo' has unpleasant effect of not allowing debugging but simple trick works: if __name__ == "__main__": import twisted.scripts.trial import sys sys.argv = ['-m', 'my_test' ] twisted.scripts.trial.run() (BTW, what about patching run so it could get parameters and fallback to argv only when they are not given?) Currently I have the following problem: my test script hangs. Via simplification I got to the following text (requires some postgres database): from twisted.trial import unittest from twisted.trial.util import deferredResult import psycopg from twisted.enterprise import adbapi DSN = "dbname=marcink port=5433 user=user password=pwd host=localhost" class MyTestCase(unittest.TestCase): def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) # (populating some test data) deferredResult( self.dbpool.runInteraction(self._insertTestData) ) def tearDown(self): deferredResult( self.dbpool.runOperation("DROP TABLE mytable") ); def _insertTestData(self, tx): tx.execute("CREATE TABLE mytable(id INTEGER)") tx.execute("INSERT INTO mytable(id) VALUES(1)") def testSomething(self): d = self.dbpool.runQuery("SELECT id FROM mytable") d.addCallback( lambda rows: self.assertEquals(len(rows), 1) ) When I run trial -v my_test I get my_test MyTestCase testSomething ... (and the test hangs forever)
Hmm, replying to myself, it seems that when I changed to def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) return self.dbpool.runInteraction(self._insertTestData) my test seems to work. Could anyone confirm that setUp (and tearDown) are also allowed to return deferreds? I have also another problem. In fact I need to run a few deferred-returning functions within setUp. How should I join them? I can create DeferredList but will it then be checked for failures?
Continuing replying to myself, the test seems to work even too well. In fact it insist on succeeding. For instance when I changed sql command to some rubbish, the test still succeeds. For example this test is raported as succesfully executed def testSomething(self): d = self.dbpool.runQuery("RUBBISH SELECT BLAH BLAH") return d I had to miss something. But ... what?
As one more attempt I just ugraded from twisted 1.3 to twisted 2.0 (= from twisted version present in Debian testing to twisted version present in Debian unstable). And now ........ the script which in 1.3 ignored errors but worked, *hangs(, in exactly the same way in which it used to hang when I used deferredResult.
I found the solution. The following modification caused everything to work as expected def setUp(self): self.dbpool = adbapi.ConnectionPool('psycopg', DSN) self.dbpool.start() # ... (I added self.dbpool.start()) Confusing part is that start method doc tells: If you are using the reactor normally, this function does *not* need to be called. So it seems trial is *not* using reactor 'normally' - whatever does it mean.
On Thu, 20 Oct 2005 18:07:19 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
Indeed. Hopefully this will be rectified at some future point, but it is difficult to fix for now, due to the way the vast majority of existing tests are written, along with the desire to retain backwards compatibility. Jp
Maybe documentation could be slightly patched instead? Some sentence suggesting that one should call 'start' method if the application seem to hang could be added to the adbapi description... I must say (and you can see on this list ;-)) that for novice twisted programmer like me the whole thing was very confusing.
On Thu, 20 Oct 2005 16:05:03 +0200, Marcin Kasperski <marcin.kasperski@softax.com.pl> wrote:
Just so you know, and in case anyone reading isn't familiar with it, trial has a debug mode. If you pass the -b flag, it will automatically set up pdb, plus a couple other Twisted-specific features that come in fairly handy. twistd also supports this, letting you break into pdb at any point while the app is running, and also stopping for unhandled errors. Jp
I found it, but for not-so-hardcore python programmer, pdb is not the tool of the dreams. GUI Debugger (for instance one included in WingIDE or in Komodo or in Eric or ... is far more useful ....). And to use one it is necessary to start script within the debugger control, not to start debugger from trial. And, as I started to talk about GUIs, it would be also nice to run unittests under GUI control (I love eric3-unittest - using it even when I am not using eric environment itself).
participants (4)
-
Jonathan Lange -
Jp Calderone -
Marcin Kasperski -
Stefano Canepa