[IPython-dev] Dropping our parametric tests system

Thomas Kluyver takowl at gmail.com
Fri Aug 30 16:38:55 EDT 2013


I propose that we get rid of our parametric test system, in favour of
either writing regular tests, or using nose's test generators.

The parametric tests have failed to run on at least three occasions:
- Old versions of nose, which we ended up monkeypatching
- Again on newer versions of nose with Python 2, which Min's PR #4148 and
my #4150 fix using monkeypatches
- On Python 3, it only runs to the first yield, and presumably has done
ever since we first had a Python 3 port (also addressed in #4150)

So it fails a lot, and it fails quietly, without any warnings or error
messages, which is exactly what you don't want for a test suite. You just
realise one day that lots of tests aren't actually being run. Finally, the
fixes that we've had to introduce are conceptually horrible, and the
ParametricTestCase code is fairly complex itself.

The advantage of parametric tests over conventional tests is mainly that
the testing output reports a greater number of tests being run, even though
it's not actually exercising any more code. The advantage over nose test
generators is that the code is executed in the real context in which you'd
want to debug it. I don't think that either benefit, and especially the
benefit over plain tests, is worth the pain this is causing us.

Min suggested that I write an IPEP for this, but in my opinion this isn't a
big enough change to require an IPEP. I'm happy to write one if people feel
I need to, though.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20130830/13337e4f/attachment.html>


More information about the IPython-dev mailing list