Re: [Twisted-Python] automated tests for a server application
On 2018-11-06 11:28, Chris Withers wrote:
I guess I'm still not clear on what the point of using a 'fake' reactor over a 'real' one is. Not that I'm an expert here...
I would assume you would just write a 'client' in the test of whatever complexity (could just write hardcoded byte sequences) which opens a connection to the server and transmits the bytes and then asserts things about the response. But no, I don't have any code. I can't say I have a good test suite myself and I also don't actually use Twisted for internet stuff (canbus and serial). Sorry. I would expect the servers provided by Twisted would have their own tests you could look at though. Cheers, -kyle
On Tue, Nov 6, 2018, at 8:43 AM, Kyle Altendorf wrote:
There are a bunch of advantages! Including: 1. Your tests become more deterministic --- even *fully* deterministic. 2. Fake I/O is way faster --- no syscalls! 3. Fake I/O is more reliable --- no worries about failing to allocate a port. 4. You gain full control over time. This makes it easy to test behaviors like timeouts and backoff timers. 5. You can run multiple fake reactors. Great for testing distributed systems! At an extreme, you can make the whole suite synchronous (meaning, Deferreds fire immediately). treq.testing[1] can be used like this. [2] is an example of a suite in this style (this suite is actually run under the Django test runner, which knows nothing of reactors or Deferreds, though I don't recommend doing that). There are some potential pitfalls: 1. As with any test double, you can trick yourself. I find faking I/O in this way is less prone to these problems than tools like unittest.mock, though. 2. Manually controlling time can harm composability. Application-level Twisted tends to eventually require some sort of time-related behavior, if only the humble reactor.callLater(0, ...) to avoid stack overflow. (Use of the cooperator from twisted.internet.task is another common one.) 3. Some (old) bits of Twisted don't allow passing a reactor or use the one passed in all code paths[3]. You can combine fake I/O with techniques like generative testing, too, though I don't have any public examples to point at. And of course you can also combine with other forms of dependency injection, like passing in an endpoint. FWIW twisted.test.proto_helpers and twisted.test.iosim *are* public API, despite their presence in the test package. They could certainly use docs, and there is a ticket to move them somewhere more appropriate. For the moment the best way to learn to read them is to look at Twisted's own test suite. I found reading the tests for HostnameEndpoint helpful [4]. ---Tom [1]: https://treq.readthedocs.io/en/latest/testing.html [2]: https://github.com/twm/yarrharr/blob/e31a488f8884fb82f1dba733585bef4a7ad8696... [3]: https://github.com/twm/yarrharr/pull/277/commits/1a5b5832edbf1fb1b1f45e9d99b... [4]: https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/test/test...
On Tue, Nov 6, 2018, at 8:43 AM, Kyle Altendorf wrote:
There are a bunch of advantages! Including: 1. Your tests become more deterministic --- even *fully* deterministic. 2. Fake I/O is way faster --- no syscalls! 3. Fake I/O is more reliable --- no worries about failing to allocate a port. 4. You gain full control over time. This makes it easy to test behaviors like timeouts and backoff timers. 5. You can run multiple fake reactors. Great for testing distributed systems! At an extreme, you can make the whole suite synchronous (meaning, Deferreds fire immediately). treq.testing[1] can be used like this. [2] is an example of a suite in this style (this suite is actually run under the Django test runner, which knows nothing of reactors or Deferreds, though I don't recommend doing that). There are some potential pitfalls: 1. As with any test double, you can trick yourself. I find faking I/O in this way is less prone to these problems than tools like unittest.mock, though. 2. Manually controlling time can harm composability. Application-level Twisted tends to eventually require some sort of time-related behavior, if only the humble reactor.callLater(0, ...) to avoid stack overflow. (Use of the cooperator from twisted.internet.task is another common one.) 3. Some (old) bits of Twisted don't allow passing a reactor or use the one passed in all code paths[3]. You can combine fake I/O with techniques like generative testing, too, though I don't have any public examples to point at. And of course you can also combine with other forms of dependency injection, like passing in an endpoint. FWIW twisted.test.proto_helpers and twisted.test.iosim *are* public API, despite their presence in the test package. They could certainly use docs, and there is a ticket to move them somewhere more appropriate. For the moment the best way to learn to read them is to look at Twisted's own test suite. I found reading the tests for HostnameEndpoint helpful [4]. ---Tom [1]: https://treq.readthedocs.io/en/latest/testing.html [2]: https://github.com/twm/yarrharr/blob/e31a488f8884fb82f1dba733585bef4a7ad8696... [3]: https://github.com/twm/yarrharr/pull/277/commits/1a5b5832edbf1fb1b1f45e9d99b... [4]: https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/test/test...
participants (2)
-
Kyle Altendorf -
Tom Most