[pypy-dev] Improved Unit Test Framework

holger krekel hpk at trillke.net
Sun Oct 19 23:43:26 CEST 2003

Hi Peter,

[Peter Hansen Sun, Oct 19, 2003 at 02:07:42PM -0400]
> holger krekel wrote:
> > 
> > [Peter Hansen Sun, Oct 19, 2003 at 09:14:43AM -0400]
> > > holger krekel wrote:
> > > > >  - improvements to unittest itself
> > >
> > > Just what would those be?  I still haven't heard a single useful
> > > improvement.  There must be some (at least, that's obviously what
> > > a lot of people feel), but what are they?
> > 
> > ok, here a few possible improvements for starters (i have a lot more in
> > mind :-)
> > 
> > - beeing able to provide a function that executes a test-method
> >   (today you have to replicate all the boilerplate in TestCase.__call__
> >   etc.pp.)
> Why do you need to do that?  Or maybe I should say, can't you already
> do that?  If you have a TestCase, I think you just call .run or something
> on it, same as when you specify a single class and (optionally) a test
> method name on the command line.  I frequently run a single test using
> our framework wrapper, which just stole code from the existing runner
> in unittest.

I don't mean selecting but executing a test.  I am really just interested in
having a TestCase hook like this:

    def execute_test(self, callable):
        """ execute a testmethod """
        callable()      # default implementation

Maybe it's special to PyPy but we could use this for our application-level 
tests.  There are two basic levels at which a test can possibly run:

- application level: such a test is executed indirectly using 
  the PyPy interpreter (which itself requires an Object Space to
  operate on objects). 

- interpreter level: those are executed at the same level as how
  the interpreter sees the world: as wrapped objects which
  only an Object Spaces knows how to access and manipulate.

Application level tests are faster and easier to write. For driving
them the above hook would probably help. 

> > - beeing able to raise e.g. TestSkip Error to indicate that a test
> >   should not be run with the current setup. (You can't alway tell
> >   that offhand in setUp).
> What does this do that raise TestSkipError (after defining it)
> would do not do?  Continue running other tests?  Seems like a 
> property of the test runner, not the unittest side of things 
> (assuming a new test runner, as we are by making the distinction 
> into two categories, I think).

OK, we could pack this to the TestRunner which resides also in unittest.py, 
anyway ...

> > - allow providing extra options to the cmdline-interface
> > 
> > - plan for/do other user-interfaces (web-driven, pygame, whatever :-)
> > 
> > - provide means (a TestConsole) to easily get a list of failed tests
> >   (interactively), keep the testing-process alive and repetitively redo
> >   the failed tests. Or save off the info to some file and reload it
> >   on rerunning the test.
> These are all properties of the test runner, aren't they?  The
> existing unittest has its own version of this stuff, but I got
> the impression you were already accepting that a different runner
> would be needed.

In some strict sense maybe only TestCase does not belong to the the
"frontend", which then includes TestRunner, TestProgram, main,
TestLoader, TestResult etc.pp.  

However, my impression is that the mechanics in unittest.py are not really
flexible enough to drive our development in a sufficiently nice way. 
And when doing test-driven development why not have *very* nice tools
(instead of OK-tools) to support it?  



More information about the Pypy-dev mailing list