[Twisted-Python] Trial Test Runner
Hi, I'm having a hard time writing a test suite using trial. What's the best documentation available? I've the API Reference as well as the sources at hand, but... For example I've tried this: #!/usr/bin/python # -*- test-case-name: eUnit.test.test_sample -*- from twisted.trial import unittest, runner, reporter class Test(unittest.TestCase): def testSample(self): print "hallo welt" if __name__ == "__main__": runner.TrialRunner(reporter.TextReporter).run(Test) It's failing with: TypeError: unbound method countTestCases() must be called with Test instance as first argument (got nothing instead) Some more documentation (an introduction or tutorial or some such) would be very welcome and helpful. Regards Markus
On Wed, 15 Nov 2006 18:09:01 +0100, Markus Schiltknecht
Hi,
I'm having a hard time writing a test suite using trial. What's the best documentation available? I've the API Reference as well as the sources at hand, but...
For example I've tried this:
#!/usr/bin/python # -*- test-case-name: eUnit.test.test_sample -*-
from twisted.trial import unittest, runner, reporter
class Test(unittest.TestCase):
def testSample(self): print "hallo welt"
if __name__ == "__main__": runner.TrialRunner(reporter.TextReporter).run(Test)
It's failing with: TypeError: unbound method countTestCases() must be called with Test instance as first argument (got nothing instead)
Some more documentation (an introduction or tutorial or some such) would be very welcome and helpful.
Trial has a command line tool for running suites, "trial". If that code is in eUnit/test/test_sample.py, then you can try any of these commands: trial eUnit trial eUnit.test trial eUnit.test.test_sample trial eUnit.test.test_sample.Test trial eUnit.test.test_sample.test.testSample trial /path/to/eUnit/ trial /path/to/eUnit/test/ trial /path/to/eUnit/test/test_sample.py You don't need any of the code in your __main__ section. Jean-Paul
Hi, Jean-Paul Calderone wrote:
Trial has a command line tool for running suites, "trial". If that code is in eUnit/test/test_sample.py, then you can try any of these commands:
Duh! Thank you! I didn't think about a command line tool and was looking in the wrong places. Another question: can I somehow nest setUp and tearDown methods? I.e. A.setUp: very general setup, i.e. starting a process needed A.test1 A.test2 A.B.setUp: specific setups for B, which depend on A's setup A.B.test1 A.B.tearDown A.test3 A.tearDown: tear down all the general stuff, i.e. stopping that process Regards Markus
On Fri, 2006-11-17 at 12:57 +0100, Markus Schiltknecht wrote:
Hi,
Jean-Paul Calderone wrote:
Trial has a command line tool for running suites, "trial". If that code is in eUnit/test/test_sample.py, then you can try any of these commands:
Duh! Thank you! I didn't think about a command line tool and was looking in the wrong places.
Another question: can I somehow nest setUp and tearDown methods? I.e.
A.setUp: very general setup, i.e. starting a process needed A.test1 A.test2
A.B.setUp: specific setups for B, which depend on A's setup A.B.test1 A.B.tearDown
A.test3 A.tearDown: tear down all the general stuff, i.e. stopping that process
You can use class inheritance. Define A deriving from object, and then B from A and TestCase, for example. -- Groetjes, ralphm
Hi, Ralph Meijer wrote:
You can use class inheritance. Define A deriving from object, and then B from A and TestCase, for example.
That only works for Fixtures which take a few milliseconds to setUp and tearDown, because for every derived TestCase, the Fixture is setUp and tornDown. That won't work in my case, because setting up the Fixture is quite complex and takes up to 20 seconds. I definitely don't want to do that for every contained TestCase. Why does the TrialSuite not have setUp and tearDown methods? Regards Markus
On Fri, 17 Nov 2006 14:22:14 +0100, Markus Schiltknecht
Hi,
Ralph Meijer wrote:
You can use class inheritance. Define A deriving from object, and then B from A and TestCase, for example.
That only works for Fixtures which take a few milliseconds to setUp and tearDown, because for every derived TestCase, the Fixture is setUp and tornDown. That won't work in my case, because setting up the Fixture is quite complex and takes up to 20 seconds. I definitely don't want to do that for every contained TestCase.
Why does the TrialSuite not have setUp and tearDown methods?
It is not intended that users will subclass TrialSuite. You can put your expensive set up code into a free function which caches its result to avoid performing redundant work, then call it from whichever setUp methods need it. Jean-Paul
Hi, Jean-Paul Calderone wrote:
It is not intended that users will subclass TrialSuite.
Okay.
You can put your expensive set up code into a free function which caches its result to avoid performing redundant work, then call it from whichever setUp methods need it.
You can not cache the startup of a process or of a network connection. Probably unit tests are not designed for such things, but I would like to use twisted as it's event based. And I want to test for events. It looks like a good fit. Ideas? Regards Markus
On Fri, 2006-11-17 at 15:26 +0100, Markus Schiltknecht wrote:
Hi,
Jean-Paul Calderone wrote:
It is not intended that users will subclass TrialSuite.
Okay.
You can put your expensive set up code into a free function which caches its result to avoid performing redundant work, then call it from whichever setUp methods need it.
You can not cache the startup of a process or of a network connection. Probably unit tests are not designed for such things, but I would like to use twisted as it's event based. And I want to test for events. It looks like a good fit.
Maybe you test too much. In general you don't need actual network connections to test protocols. For example, there are helpers in twisted.test.proto_helpers for mimicking a transport. -- Groetjes, ralphm
Hi, Ralph Meijer wrote:
Maybe you test too much. In general you don't need actual network connections to test protocols. For example, there are helpers in twisted.test.proto_helpers for mimicking a transport.
I want to test for example, how a program (written in C) reacts to network delays or outages. I absolutely need (and already have) python code to emulate the network layer. Of course I could test just one instance of the program, but then I'd have to write a dummy peer, just for the tests. Instead, it seems a lot easier to use the real code itself and check if two processes correctly talk to each other. It's a Multi-Master protocol. I would like to simulate some of the dirty stuff that can happen to an application out in the wild (OOM kill, network delays and outages, etc..). Another requirement I'm facing is benchmarks. How well does the application cope with different a packet loss rate of 10% compared to no packet losses? So, if you tell me I want to do to much with unit tests, that's fine, I don't need to run unit tests. They are probably not designed for what I want to do. But what else can I use? Is there anything you know which could possibly fulfill these requirements? IMO, twisted itself looks very nice in that it is event-based and provides very good interfaces to network protocols as well as managing sub-processes. Perhaps I just need to write my own test runners on top of twisted and forget about trial. OTOH, wouldn't it be sufficient to add setUp and tearDown methods for TrialSuites? I'm investigating into that direction. Regards Markus
Hi, Markus Schiltknecht wrote:
OTOH, wouldn't it be sufficient to add setUp and tearDown methods for TrialSuites? I'm investigating into that direction.
What speaks against something like that: Index: runner.py =================================================================== --- runner.py (revision 18715) +++ runner.py (working copy) @@ -142,10 +142,14 @@ """ # we implement this because Python 2.3 unittest defines this code # in __call__, whereas 2.4 defines the code in run. + if hasattr(self, 'setUp'): + self.setUp(self) for test in self._tests: if result.shouldStop: break test(result) + if hasattr(self, 'tearDown'): + self.tearDown(self) return result ..and adding a setUp and tearDown method in the loader if the module has such methods? Regards Markus
On Fri, 17 Nov 2006 15:26:02 +0100, Markus Schiltknecht
Hi,
Jean-Paul Calderone wrote:
It is not intended that users will subclass TrialSuite.
Okay.
You can put your expensive set up code into a free function which caches its result to avoid performing redundant work, then call it from whichever setUp methods need it.
You can not cache the startup of a process or of a network connection.
This may be a trial limitation which needs to be addressed. On the other hand, unit tests which involve processes and the network are not as good as unit tests which don't involve such things.
Probably unit tests are not designed for such things, but I would like to use twisted as it's event based. And I want to test for events. It looks like a good fit.
Ideas?
It sounds like you're not really writing a unit test suite. What are you writing? Jean-Paul
Hi, Jean-Paul Calderone wrote:
It sounds like you're not really writing a unit test suite. What are you writing?
Yeah, maybe I do not write unit tests. I want to check a C program which forks, if it behaves correctly when sub-processes get killed or if it recovers correctly from delayed or lost network connections. I've been looking at other automated testing frameworks, like STAF, but it does not allow me to write such simple tests... Regards Markus
participants (3)
-
Jean-Paul Calderone -
Markus Schiltknecht -
Ralph Meijer