On Wed, 2010-01-20 at 22:32 -0500, Mark Roddy wrote:
I think that this is the wrong approach to the problem: - class scope for such fixtures drives large test classes, reduces flexability
I can see how this could arise, but it has not been my experience (I have a class setup system I've been using for a little while). A setUp method alone can be abused to create large inflexible tests. I think the answer to this is good documentation that clearly states that abusing the functionality can lead to shooting oneself in the foot.
Yes, setUp and tearDown can be problematic as well. Junit which has has setUp and setUpClass a long time has recently (http://kentbeck.github.com/junit/doc/ReleaseNotes4.7.html) added a system called Rules which is similar to testresources (though the fine detail is different). The key elements are the same though: - rules are their own hierarchy - rules are added to tests not part of the test specific code - framework takes care of bringing up and taking down rules.
- doing it the rough way you suggest interacts in non obvious ways with test order randomisation
This is dependent on how randomization is implemented. Something simple such as pick a random test of those available shouldn't affect the ability to keep track of whether a class's setup method hasn't been implemented, but imitadly I'm not familiar with the internals of any existing randomization system so I will look into these to see what issues that would arise from this change.
It can be made to work: the non-obviousness is a simple side effect of having dependencies between multiple tests.
- it also interacts with concurrent testing by having shared objects (classes) hold state in a way that is not introspectable by the test running framework.
Very true, but it seems counter intuitive to expect to be able to have shared state in this context. At the same time, it wouldn't be acceptable to change in a way that breaks frameworks with such functionality.
I don't think shared state here is good: but setting attributes *on the class* *is* shared state: Its why I don't like setUpClass :).
I'd much much much rather see e.g. testresources integrated into the core allowing fixtures to be shared across test instances in a way that doesn't prohibit their use with concurrent testing, doesn't make it awkward to do it across multiple classes. I'm happy to make any [reasonable] changes (including license) to testresources to make it includable in the stdlib if that's of interest.
A pretty good approach for a complicated setup, but in a simple case where you only need a line or two it seems like a lot of boiler plate to get the job done. Though I'll look into this library a little more as I am not intimately familiar with it at the moment.
If you only need a line or two, its hard to justify setUpClass being used :). Anyhow, its really not much boilerplate: class MyResource(TestResource): def make(self, deps): # line or two return thing I am considering adding a helper to testresources: def simple_resource(maker, cleaner): class MyResource(TestResource): def make(self, deps): return maker(deps) def clean(self, resource): cleaner(resource) return MyResource which would make the boilerplate smaller still. Cheers, Rob