[Distutils] Draft PEP for JSON based metadata published

[holger krekel]

On Wed, May 29, 2013 at 16:26 -0400, Daniel Holth wrote:
> On Wed, May 29, 2013 at 3:33 PM, holger krekel <holger at merlinux.eu> wrote:
> > On Wed, May 29, 2013 at 15:15 -0400, Daniel Holth wrote:
> >> On Wed, May 29, 2013 at 2:36 PM, holger krekel <holger at merlinux.eu> wrote:
> >> > Hi Nick,
> >> >
> >> > On Mon, May 27, 2013 at 21:36 +1000, Nick Coghlan wrote:
> >> >> After preliminary reviews by Donald and Daniel, I have now pushed the
> >> >> first complete draft of the JSON-based metadata 2.0 proposal to
> >> >> python.org
> >> >>
> >> >> PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/
> >> >
> >> > After a first quick read i am wondering if i missed something
> >> > with respect to test tools.  There are some fields which specify
> >> > dependencies required for running tests, but there is nothing that
> >> > would tell which test runner to use, which test command to invoke,
> >> > or am i missing something?
> >> >
> >> > Basically i am wondering how PEP426 could be useful/used by tox.
> >> > best,
> >>
> >> The first thing we might do is to have setuptools expose its
> >> test_suite argument as "extensions" : { "setuptools": { "test_suite":
> >> "callable.name" } }.
> >
> > The way tox specifies testing is to allow arbitrary test commands not
> > just unittest test suites.  People use py.test, make and whatnot
> > to run tests.
> 
> tox could shove a JSON version of its config into the metadata as an
> extension.
>
> If "something wrapped in a unittest suite" isn't the interface for
> running tests (in the most boring way possible, with better runners
> for the developers not the users of the package), then what should it
> be?

An entry point or way to execute a command line script, either coming
from an test_requires dependency, or a script contained in the package.

Note that py.test can run unittest test suites, but pytest test suites
are not modeled as a unittest test suite.  The same would be true if
you use a test script contained in a package or "make" for that matter.

IOW, i think there should be a field denoting which command line tool
or script to run for the tests.  If it's not defined and a test suite
is defined, it's of course fine to invoke unitest's machinery on it.

> >> I think we need the next version of the metadata or sdist 2.0 to
> >> really do a better job than just running "setup.py test"; right now
> >> the tests usually only make sense in the context of an unpacked sdist.
> >> Input appreciated.
> >
> > We certainly don't want to advocate using "setup.py" for running tests as
> > we want to get way from the neccessity for this file to exist (correct me
> > if i am wrong with this presumption).  The metadata should be rich
> > enough to support tools like tox to perform the testing, much
> > like pip for installations.
> 
> The first priority is to get rid of "setup.py install" since that is
> where most of the trouble comes from. It's not that big of a task to
> relegate setup.py to its proper place as a build script. At the same
> time it will eventually become possible to use non-distutils-derived
> build tools without having to emulate a distutils setup.py.
> 
> Wouldn't we continue to type tox, or nose, or py.test, with the
> standard hook being mainly for automated tools that just want to
> verify that something still works before building / installing?

Typically projects depend on the test runner/framework for executing
their tests, be it at development or pre-installation time.

holger