[Python-ideas] Test Class setup and teardown in unittest

[Robert Collins]

On Wed, 2010-01-20 at 22:32 -0500, Mark Roddy wrote:

> > I think that this is the wrong approach to the problem:
> >  - class scope for such fixtures drives large test classes, reduces
> > flexability

> I can see how this could arise, but it has not been my experience (I
> have a class setup system I've been using for a little while).  A
> setUp method alone can be abused to create large inflexible tests.  I
> think the answer to this is good documentation that clearly states
> that abusing the functionality can lead to shooting oneself in the
> foot.

Yes, setUp and tearDown can be problematic as well. Junit which has has
setUp and setUpClass a long time has recently
(http://kentbeck.github.com/junit/doc/ReleaseNotes4.7.html) added a
system called Rules which is similar to testresources (though the fine
detail is different). The key elements are the same though:
 - rules are their own hierarchy
 - rules are added to tests not part of the test specific code
 - framework takes care of bringing up and taking down rules.

> >  - doing it the rough way you suggest interacts in non obvious ways with
> > test order randomisation

> This is dependent on how randomization is implemented.  Something
> simple such as pick a random test of those available shouldn't affect
> the ability to keep track of whether a class's setup method hasn't
> been implemented, but imitadly I'm not familiar with the internals of
> any existing randomization system so I will look into these to see
> what issues that would arise from this change.

It can be made to work: the non-obviousness is a simple side effect of
having dependencies between multiple tests.

> >  - it also interacts with concurrent testing by having shared objects
> > (classes) hold state in a way that is not introspectable by the test
> > running framework.

> Very true, but it seems counter intuitive to expect to be able to have
> shared state in this context.
> At the same time, it wouldn't be acceptable to change in a way that
> breaks frameworks with such functionality.

I don't think shared state here is good: but setting attributes *on the
class* *is* shared state: Its why I don't like setUpClass :).

> > I'd much much much rather see e.g. testresources integrated into the
> > core allowing fixtures to be shared across test instances in a way that
> > doesn't prohibit their use with concurrent testing, doesn't make it
> > awkward to do it across multiple classes. I'm happy to make any
> > [reasonable] changes (including license) to testresources to make it
> > includable in the stdlib if that's of interest.

> A pretty good approach for a complicated setup, but in a simple case
> where you only need a line or two it seems like a lot of boiler plate
> to get the job done.  Though I'll look into this library a little more
> as I am not intimately familiar with it at the moment.

If you only need a line or two, its hard to justify setUpClass being
used :).

Anyhow, its really not much boilerplate:
class MyResource(TestResource):
    def make(self, deps):
        # line or two
        return thing

I am considering adding a helper to testresources:

def simple_resource(maker, cleaner):
    class MyResource(TestResource):
        def make(self, deps):
            return maker(deps)
        def clean(self, resource):
            cleaner(resource)
    return MyResource

which would make the boilerplate smaller still.

Cheers,
Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20100121/f9b807e6/attachment.pgp>