On 6/7/07, Alex Lang firstname.lastname@example.org wrote:
Here is a problem regarding the distributed trial test runner. The slave instances receive data from the master, telling them which tests to run. This requires use of the reactor on the slave-side. Unfortunately, when the testCase does the cleanup of the reactor, it cleans up everything including the trial slave.
Any ideas on what should be done about this?
This is a roundabout way of answering your question, but it's helpful for me to write it down, and I think it will inform the discussion.
In terms of Trial's requirements: * We want to write and run tests that perform asynchronous operations, which essentially means tests that return Deferreds. * We want to be able to run these tests in non-Trial test runners. * We want tests to be isolated as much as possible. * We want to be able to use Trial to test the reactor itself.
Some awkward facts: * There is exactly one reactor per Twisted process. This reactor cannot be cleanly restarted in a cross-platform way. * The current Twisted test suite (i.e. what is loaded when you do 'trial twisted') does not play nicely with the reactor. Changing this will take a significant amount of work.
So, in response to all of these things: * Trial presents a blocking interface for running any given test: TestCase.run(). This allows Deferred-returning tests to be used in non-Twisted test runners. * By design, Trial totally obliterates the reactor in each call to TestCase.run(). This helps ensure test isolation * Trial's design operates on the assumption that the caller of TestCase.run() will not be doing anything with the reactor.
However, all of this presents problems: * The vast bulk of Twisted's code base cannot be used to implement a Trial runner. This is what Alex is discovering now. * Trial's cleanup code is buggy and fragile. Changing anything is likely to break hundreds of tests. * Tests that use twisted.trial.unittest.TestCase run in a highly-specialized, somewhat unrealistic environment.
Various solutions have been proposed: * Using a customized reactor wrapper in trial that tracks resources created by tests and cleans up only those resources. * Making reactors restartable. * Allowing for multiple running reactors. i.e. A reactor that could be started within another running reactor.
I believe that only the third of these solutions would actually help solve Alex's problem. However, as I understand it, the solution would require extensive API changes across most of Twisted, as things which once assumed a global reactor must now take 'reactor' as a parameter.
I'd rather not offer any recommendations in this email. It's difficult enough figuring out what the problem is and what the options are.
 For example, Trial does two calls to 'reactor.iterate()' in its cleanup. If these calls are removed, tests fail and there are a significant number of cleanup errors. I discussed some of the issues here in an email last year.