[Distutils] Where should I put tests when packaging python modules?

David Wilson dw+distutils-sig at hmmz.org
Wed Oct 7 01:14:24 CEST 2015

On Tue, Oct 06, 2015 at 09:51:01AM +0200, Antoine Pitrou wrote:

> They should be inside the module. That way, you can check an installed
> module is ok by running e.g. "python -m mypackage.tests". Any other
> choice makes testing installed modules more cumbersome.

As Donald mentioned, this doesn't work in the general case since many
packages ship quite substantial test data around that often doesn't end
up installed, and in other cases since the package requires significant
fixture setup or external resources (e.g. running SQLAlchemy tests
without a working database server would be meaningless).

The option of always shipping test data as a standard part of a package
in a vein attempt to always ensure it can be tested (which is not always
likely given the SQLAlchemy example above) strikes me as incredibly
wasteful, not from some oh-precious-bytes standpoint, but from the
perspective of distributing a Python application of any size where the
effects of always shipping half-configured test suites has increased the
resulting distribution size potentially by 3 or 4x.

https://github.com/bennoleslie/pexif is the first hit on Google for a
module I thought would need some test data. It's actually quite
minimally tested, yet already the tests + data are 3.6x the size of the
module itself.

I appreciate arguments for inlining tests alongside a package in order
to allow reuse of the suite's functionality by consuming applications'
test suites, but as above, in the general case this simply isn't
something that will always work and can be relied on by default.

Is there perhaps a third option that was absent from the original post?
e.g. organizing tests in a separate, optional, potentially
pip-installable package.

> > outside the module like this:
> > 
> >  https://github.com/pypa/sampleproject/tree/master/tests
> There is no actual reason to do that except win a couple kilobytes if
> you are distributing your package on floppy disks for consumption on
> Z80-based machines with 64KB RAM.
> Even Python *itself* puts its test suite inside the standard library,
> not outside it (though some Linux distros may strip it away).
> Try "python -m test.regrtest" (again, this may fail if your distro
> decided to ship the test suite in a separate package).
> The PyP"A" should definitely fix its sample project to reflect good
> practices.
> Regards
> Antoine.
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig

More information about the Distutils-SIG mailing list