On 6/7/07, Alex Lang alex.lang@mail.mcgill.ca wrote:
Hello,
Here is a problem regarding the distributed trial test runner. The slave instances receive data from the master, telling them which tests to run. This requires use of the reactor on the slave-side. Unfortunately, when the testCase does the cleanup of the reactor, it cleans up everything including the trial slave.
Any ideas on what should be done about this?
This is a roundabout way of answering your question, but it's helpful for me to write it down, and I think it will inform the discussion.
In terms of Trial's requirements: * We want to write and run tests that perform asynchronous operations, which essentially means tests that return Deferreds. * We want to be able to run these tests in non-Trial test runners. * We want tests to be isolated as much as possible. * We want to be able to use Trial to test the reactor itself.
Some awkward facts: * There is exactly one reactor per Twisted process. This reactor cannot be cleanly restarted in a cross-platform way. * The current Twisted test suite (i.e. what is loaded when you do 'trial twisted') does not play nicely with the reactor. Changing this will take a significant amount of work.[1]
So, in response to all of these things: * Trial presents a blocking interface for running any given test: TestCase.run(). This allows Deferred-returning tests to be used in non-Twisted test runners. * By design, Trial totally obliterates the reactor in each call to TestCase.run(). This helps ensure test isolation * Trial's design operates on the assumption that the caller of TestCase.run() will not be doing anything with the reactor.
However, all of this presents problems: * The vast bulk of Twisted's code base cannot be used to implement a Trial runner. This is what Alex is discovering now. * Trial's cleanup code is buggy and fragile. Changing anything is likely to break hundreds of tests. * Tests that use twisted.trial.unittest.TestCase run in a highly-specialized, somewhat unrealistic environment.
Various solutions have been proposed: * Using a customized reactor wrapper in trial that tracks resources created by tests and cleans up only those resources. * Making reactors restartable. * Allowing for multiple running reactors. i.e. A reactor that could be started within another running reactor.
I believe that only the third of these solutions would actually help solve Alex's problem. However, as I understand it, the solution would require extensive API changes across most of Twisted, as things which once assumed a global reactor must now take 'reactor' as a parameter.
I'd rather not offer any recommendations in this email. It's difficult enough figuring out what the problem is and what the options are.
jml
[1] For example, Trial does two calls to 'reactor.iterate()' in its cleanup. If these calls are removed, tests fail and there are a significant number of cleanup errors. I discussed some of the issues here in an email last year.