Re: [Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them

At 11:19 AM 2/26/2010 -0500, Olemis Lang wrote:

On Fri, Feb 26, 2010 at 10:52 AM, P.J. Eby wrote:

...

At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote:

...

Well not as happily as I thought in first place. The fact is that `setuptools` test command creates the runner by calling the class specified in `test_runner` arg but supplies no parameters. In this case I'd like to specify `xmlrunner:XMLTestRunner` (is that the correct syntax ?) to obtain a JUnit XML report, but I need to specify the path where those reports will be outputted, so that the CI tool (Hudson in this case) can publish'em and generate the nice graphs we all love . Considering the current implementation, I still need to override `run_tests` method as a whole, and repeat 99% of what is implemented in there, in order to get the exact runner I need

:(

My suggestion (hint, $0.02 ... whatever ;o) is

1. use a factory method (e.g `create_runner` ) returning an instance of test runner. This method could be overridden by subclasses if further options or whatever needs to be considered to instanciate the runner

I'm confused. Why don't you just set test_runner to point to your factory function, then? That's what nose does. (Well, in its case it's the test_loader, but same basic idea.)

Well in theory it is possible (and did not think about that in the first place, thnx ;o) but :

1. How could I know about extra command line params (without parsing them once again of course ;o) ? 2. In the case of the build script, I'm writing a setup script rather than a framework (e.g. nose). 2.1 The XML runner is not always used, but only when running at the CI buildenv (I supposed I could just use `-r xmlrunner:XMLTestRunner` but I need an extra arg) How could I specify output path in cmdline ?

An environment variable would solve these problems, at least if you have a decent shell. ;-)

2.2 So I suppose I should write a function to do that but - Under tests pkg? ... (Unit | functional | ...) tests don't need it - In setup.py itself ? ... What should I supply in to -r arg ?

If you are running setup.py from the command line (as opposed to programmatically via easy_install), then it's __main__:whatever. (It won't work if setup.py isn't the "true" __main__, though, e.g., if the script is being execfile'd.)

How could I specify output path in cmdline ? 3. In the case of `dutest` well ... that could help (depending on the answer to 1 ;o) 3.1 How could I reuse features already stored in `Distribution` (especially values set after finalize_xxx ) ?

You can't, unless you...

4. (Hypothetical so far, so I'm prepared to receive tomatoes ;o) What if I want to reuse (subclass) `test` command ?

Which is perfectly allowable. You'll just have to duplicate that one method. Of course, if you write a suitable patch to refactor it to call another method, I'll happily put it in the trunk, since you're working from trunk to get the test_runner feature anyway. ("Suitable" = tested by you, doesn't break or remove existing functionality, & other "I'll know it if I see it" criteria ;-) )