[Distutils] RFC: Standard Declaration of tests in eggs

Jim Fulton jim at zope.com
Mon Jan 22 16:14:57 CET 2007

Phillip J. Eby wrote:
> At 01:11 PM 1/5/2007 -0500, Jim Fulton wrote:
>> Phillip J. Eby wrote:
>> ...
>>>> Why can't an entry point invoke a test loader itself?
>>>> This seems much simpler and more straightforward to me.
>>> Because that requires you to write code for something that can 
>>> adequately be expressed through an existing configuration mechanism.
>>> *And* you have to write that code in every project.
>> It's probably a couple of lines. Import a loader and call it.
>> That doesn't seem like a big deal to me. Nice and explicit too.
> And it's enormously repetitive if you have a lot of projects.
>> Besides, I expect most projects will have a single test suite
>> (function) that could just be named directly.
> You expect wrongly.  :)

I can only speak based on my own experience.  Projects like Zope
and ZODB certainly want some sort of tool to find and assemble tests.
Eggs are allowing and encouraging us to move away from such large projects.
As we break up Zope into  individual packages, I'm finding that the individual
packages have single test suites.

>  'nose' and 'py.test' are popular *precisely 
> because* they do NOT require this.  Setuptools changed to emulate them 
> by adding ScanningLoader, even though it originally did the "test suite 
> function" thing you're proposing.
> See, I originally wrote all my test suites the way you're proposing, but 
> I changed because this is one of those "convention beats configuration" 
> situations.   It's easier to just conform to a policy and have a tool
> that applies the policy.  Different projects may have different 
> policies, so providing an option for the loader, allows them to follow 
> it.  (And it made my life easier, too, as I began doing more small 
> projects instead of a few large ones.)
> And, if you want your approach to be widely adopted, it would be best to 
> allow projects to follow their own policies without requiring them to 
> add any code, even if it's a one-liner.

They'll be adding code one way or another. Either they add it to
their test modules, or they add it to setup.py.  The advantage of
the former is that it provides more flexibility without requiring
any more work. It is also more explicit.

> So, *technically*, there may be no reason to do this, but from a 
> usability, friendliness, compatiblity, marketing, adoption, etc. format 
> I don't think the standard will be successful without allowing this 
> out.  People often have heavy psychological investment in existing 
> tools, and it is hundreds of times easier to rationalize adding a line 
> of configuration to setup() than it is to rationalize adding two or 
> three lines of code.  One is merely a packaging change, the other is a 
> programming change.

We disagree.

I don't have any problem with having tools that can be used to find
and assemble tests into a test suite.  I find the unittest loader
framework to be baroque. I'd rather let people use whatever framework
they want to do this and to control this from Python.

I withdraw the proposal.  I wouldn't object to the proposal you want,
if you want to write it.  I don't have the time or interest to write
it myself.


Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

More information about the Distutils-SIG mailing list