[Distutils] RFC: Barely minimal CI + tests + packaging in Python WAS: Guidelines for where to put tests & how to package them

[Olemis Lang]

On Fri, Feb 26, 2010 at 10:52 AM, P.J. Eby <pje at telecommunity.com> wrote:
> At 10:16 AM 2/26/2010 -0500, Olemis Lang wrote:
>>
>> Well not as happily as I thought in first place. The fact is that
>> `setuptools` test command creates the runner by calling the class
>> specified in `test_runner` arg but supplies no parameters. In this
>> case I'd like to specify `xmlrunner:XMLTestRunner` (is that the
>> correct syntax ?) to obtain a JUnit XML report, but I need to specify
>> the path where those reports will be outputted, so that the CI tool
>> (Hudson in this case) can publish'em and generate the nice graphs we
>> all love . Considering the current implementation, I still need to
>> override `run_tests` method as a whole, and repeat 99% of what is
>> implemented in there, in order to get the exact runner I need
>>
>> :(
>>
>> My suggestion (hint, $0.02 ... whatever ;o) is
>>
>>   1. use a factory method (e.g `create_runner` ) returning an
>>     instance of  test runner. This method could be overridden by
>>     subclasses if further options or whatever needs to be considered
>>     to instanciate the runner
>
> I'm confused.  Why don't you just set test_runner to point to your factory
> function, then?  That's what nose does.  (Well, in its case it's the
> test_loader, but same basic idea.)
>

Well in theory it is possible (and did not think about that in the
first place, thnx ;o) but :

  1. How could I know about extra command line
      params (without parsing them once again of course ;o) ?
  2. In the case of the build script, I'm writing a setup script rather than
      a framework (e.g. nose).
     2.1 The XML runner is not always used, but only when running at
           the CI buildenv (I supposed I could just use
           `-r xmlrunner:XMLTestRunner` but I need an extra arg)
           How could I specify output path in cmdline ?
     2.2 So I suppose I should write a function to do that but
            - Under tests pkg? ... (Unit | functional | ...) tests
don't need it
            - In setup.py itself ? ... What should I supply in to -r arg ?
              How could I specify output path in cmdline ?
  3. In the case of `dutest` well ... that could help (depending on
      the answer to 1 ;o)
     3.1 How could I reuse features already stored in `Distribution`
            (especially values set after finalize_xxx ) ?
  4. (Hypothetical so far, so I'm prepared to receive tomatoes ;o)
      What if I want to reuse (subclass) `test` command ?

-- 
Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/
Blog EN: http://simelo-en.blogspot.com/

Featured article:
Added unit test for general provider error, and fixing a missing
import discovered by test. A...  -
http://bitbucket.org/osimons/trac-rpc-mq/changeset/409eb5ecdbd8/