[Distutils] RFC: Standard Declaration of tests in eggs
jim at zope.com
Fri Jan 5 18:24:10 CET 2007
Phillip J. Eby wrote:
> At 08:46 AM 1/5/2007 -0500, Jim Fulton wrote:
>> Here is a rough draft proposal for declaring tests in eggs:
>> Software packages should have automated tests. Consumers of
>> packages will often want to run these tests. Tools should be able to
>> do this automatically. This proposal seeks to provide a way for
>> automated tools to discover tests in distributions, including eggs, so
>> that tests can be run or so that test runners can be automatically
>> created to run the tests.
>> This proposal aims to be extremely simple. It has 2 parts:
>> 1. A 'test_suite' entry point is defined. An egg can provide zero or
>> more test_suite entry points. These entry points will define
>> callable objects that can be called without arguments and that
>> return unittest test suites.
>> 2. An optional 'tests' extra is defined. When creating test runners
>> or dynamically loading distributions to load tests, any
>> distributions listed in extra requires for the 'tests' extra shall
>> be included in the working set for the test runner.
> Point #2 is unnecessary, since individual entry points can list extras
> in square brackets following the module/attribute information. When
> loading an entry point, these extras automatically get require()'d.
> Conversely, if the test runner wants to manage the loading process, it
> can simply inspect the entry point object to determine the names of the
Ah, good point. Man, setuptools is soooooo rich.
> Regarding point #1, I'm not sure this is enought to define what's
> necessary. For example, it should perhaps be stated that the entry
> point must be in code that will be installed by either the distribution
> itself, or that is included in the code provided by the entry point's
> Also, an egg can't provide more than one entry point with the same name,
> so rather than a 'test_suite' entry point, we probably want an entry
> point *group*, perhaps something like 'installed.test_suites'.
That's what I meant to say. I'll clarify.
> Next, there needs to be reasonable support for dynamic test discovery,
> such as setuptools' own ScanningLoader. I would suggest that there be
> an optional entry point for the test loader class to be used to process
> the other entry points' target objects. The standard unittest protocol
> for loadTestsFromName() takes two arguments: a name and an object.
> Passing an empty string for the name, and the object loaded by the test
> suite entry point, suffices to enable normal behavior for loaders such
> as ScanningLoader, and I believe 'nose' as well as any other
> well-behaved unittest extensions.
> However, I think the spec should try to define what "well-behaved" means
> in terms of I/O, result reporting, etc. (Obviously, one requirement is
> that the test loader must be able to take an empty name string and an
> object in its loadTestsFromName() method, and return a test suite.)
Why can't an entry point invoke a test loader itself?
This seems much simpler and more straightforward to me.
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the Distutils-SIG