[Twisted-Python] Testing function with delayed calls
Hi I have a function that uses callLater extensively to schedule a number of different tasks. I don't want to get into the rationale behind such a design, but here is a contrived example which will help me explain my problem: def b(): '''Do some work''' def c(): '''Do some more work''' def a(flag): if flag: return Reactor.callLater(300, b) else: return Reactor. callLater(100, c) Now I want to test this function. Of course I can't wait for 5 minutes to ensure that `b` or `c` will indeed be called. What I need is some sort of mock clock which lets me fast forward time. Does any such thing exist in Twisted / Trial? Or is there any other approach to test such code? Regards Shakkhar
On Thu, Oct 1, 2015 at 1:40 PM, Mashiat Sarker Shakkhar < mashiat.sarker@gmail.com> wrote:
Yes there is! `MemoryReactorClock.advance` https://twistedmatrix.com/documents/current/api/twisted.test.proto_helpers.M... does this for you. An example usage can be seen in the tests for `HostnameEndpoint` https://github.com/twisted/twisted/blob/trunk/twisted/internet/test/test_end... -Ashwini
Or is there any other approach to test such code?
On Thu, Oct 1, 2015 at 5:46 PM, Ashwini Oruganti <ashfall@twistedmatrix.com> wrote:
New application code should prefer to pass and accept the reactor as a
HI Ashwini Thanks a lot for your response. I did come across MemoryReactorClock before, but I could not figure out how to use it to test my code. I am using trial as the test runner, which allows me to pass the name of the reactor to be used in command line. But MemoryReactorClock is not available, because it is not in twisted.internet. The code that I am testing imports reactor from twisted.internet in the global scope. Do I have to change that and pass a reactor explicitly (say, as a function parameter) where it is needed? Is there any other way to change the default reactor to MemoryReactorClock just for running tests? Going with my previous example, do I have to rewrite the code like following or is there a better way? def b(): '''Do some work''' def c(): '''Do some more work''' def a(myreactor, flag): if flag: return myreactor.callLater(300, b) else: return myeactor. callLater(100, c) I do see this in the documentation: parameter where it is needed, rather than relying on being able to import this module to get a reference. This simplifies unit testing ... But I'd prefer not to change the signature of existing functions. Regards Shakkhar
Indeed, the MemoryReactorClock reactor is not suitable for installation in a global reactor.
The code that I am testing imports reactor from twisted.internet in the global scope. Do I have to change that and pass a reactor explicitly (say, as a function parameter) where it is needed?
Yes.
Is there any other way to change the default reactor to MemoryReactorClock just for running tests?
You can hack it using patch <http://twistedmatrix.com/documents/15.2.0/api/twisted.trial.unittest.Synchro... <http://twistedmatrix.com/documents/15.2.0/api/twisted.trial.unittest.SynchronousTestCase.html#patch>>, by replacing the "reactor" attribute on your module. But this is much worse than passing a parameter, and should only be used if you _really_ can't change the signature of your code for some reason.
Going with my previous example, do I have to rewrite the code like following or is there a better way?
You should rewrite code like the following, it would be better :). But hey, this is Python, nobody can stop you from doing anything you want :).
Passing parameters to functions that need them is not a bad thing. You should generally prefer that. It seems a lot of Python programmers are very resistant to passing arguments, and would instead prefer that everything be a global variable. I am honestly somewhat confused as to why :-). But it's a bad habit and you should try to shake it. -glyph
On Thu, Oct 1, 2015 at 1:40 PM, Mashiat Sarker Shakkhar < mashiat.sarker@gmail.com> wrote:
Yes there is! `MemoryReactorClock.advance` https://twistedmatrix.com/documents/current/api/twisted.test.proto_helpers.M... does this for you. An example usage can be seen in the tests for `HostnameEndpoint` https://github.com/twisted/twisted/blob/trunk/twisted/internet/test/test_end... -Ashwini
Or is there any other approach to test such code?
On Thu, Oct 1, 2015 at 5:46 PM, Ashwini Oruganti <ashfall@twistedmatrix.com> wrote:
New application code should prefer to pass and accept the reactor as a
HI Ashwini Thanks a lot for your response. I did come across MemoryReactorClock before, but I could not figure out how to use it to test my code. I am using trial as the test runner, which allows me to pass the name of the reactor to be used in command line. But MemoryReactorClock is not available, because it is not in twisted.internet. The code that I am testing imports reactor from twisted.internet in the global scope. Do I have to change that and pass a reactor explicitly (say, as a function parameter) where it is needed? Is there any other way to change the default reactor to MemoryReactorClock just for running tests? Going with my previous example, do I have to rewrite the code like following or is there a better way? def b(): '''Do some work''' def c(): '''Do some more work''' def a(myreactor, flag): if flag: return myreactor.callLater(300, b) else: return myeactor. callLater(100, c) I do see this in the documentation: parameter where it is needed, rather than relying on being able to import this module to get a reference. This simplifies unit testing ... But I'd prefer not to change the signature of existing functions. Regards Shakkhar
Indeed, the MemoryReactorClock reactor is not suitable for installation in a global reactor.
The code that I am testing imports reactor from twisted.internet in the global scope. Do I have to change that and pass a reactor explicitly (say, as a function parameter) where it is needed?
Yes.
Is there any other way to change the default reactor to MemoryReactorClock just for running tests?
You can hack it using patch <http://twistedmatrix.com/documents/15.2.0/api/twisted.trial.unittest.Synchro... <http://twistedmatrix.com/documents/15.2.0/api/twisted.trial.unittest.SynchronousTestCase.html#patch>>, by replacing the "reactor" attribute on your module. But this is much worse than passing a parameter, and should only be used if you _really_ can't change the signature of your code for some reason.
Going with my previous example, do I have to rewrite the code like following or is there a better way?
You should rewrite code like the following, it would be better :). But hey, this is Python, nobody can stop you from doing anything you want :).
Passing parameters to functions that need them is not a bad thing. You should generally prefer that. It seems a lot of Python programmers are very resistant to passing arguments, and would instead prefer that everything be a global variable. I am honestly somewhat confused as to why :-). But it's a bad habit and you should try to shake it. -glyph
participants (3)
-
Ashwini Oruganti -
Glyph Lefkowitz -
Mashiat Sarker Shakkhar