[Distutils] Where should I put tests when packaging python modules?

[Wichert Akkerman]

Also having test code in the package can be very painful if you use tools like venusian which scan and try to import all Python files. 


Sent from my iPhone

> On 09 Oct 2015, at 16:05, Michael Merickel <mmericke at gmail.com> wrote:
> 
> At the pylons project we've had a history of keeping our tests inside the packages. However, keeping them outside has proven to be nicer in some projects as well. 1) It reduces the size of the binary wheels that do not need to package the tests. 2) It still allows you to run the tests on an arbitrary installation if you want to by pulling down the repo and running them against the version installed in the virtualenv. Distributing the tests to every single installation is definitely not a requirement in order to be able to run them for people who want to run them, and decoupling them can really help with that. Projects that are shipping tests inside the package may have them removed upstream and then it is very difficult to run the suite against some binary distribution.
> 
>> On Tue, Oct 6, 2015 at 6:14 PM, David Wilson <dw+distutils-sig at hmmz.org> wrote:
>> On Tue, Oct 06, 2015 at 09:51:01AM +0200, Antoine Pitrou wrote:
>> 
>> > They should be inside the module. That way, you can check an installed
>> > module is ok by running e.g. "python -m mypackage.tests". Any other
>> > choice makes testing installed modules more cumbersome.
>> 
>> As Donald mentioned, this doesn't work in the general case since many
>> packages ship quite substantial test data around that often doesn't end
>> up installed, and in other cases since the package requires significant
>> fixture setup or external resources (e.g. running SQLAlchemy tests
>> without a working database server would be meaningless).
>> 
>> The option of always shipping test data as a standard part of a package
>> in a vein attempt to always ensure it can be tested (which is not always
>> likely given the SQLAlchemy example above) strikes me as incredibly
>> wasteful, not from some oh-precious-bytes standpoint, but from the
>> perspective of distributing a Python application of any size where the
>> effects of always shipping half-configured test suites has increased the
>> resulting distribution size potentially by 3 or 4x.
>> 
>> https://github.com/bennoleslie/pexif is the first hit on Google for a
>> module I thought would need some test data. It's actually quite
>> minimally tested, yet already the tests + data are 3.6x the size of the
>> module itself.
>> 
>> 
>> I appreciate arguments for inlining tests alongside a package in order
>> to allow reuse of the suite's functionality by consuming applications'
>> test suites, but as above, in the general case this simply isn't
>> something that will always work and can be relied on by default.
>> 
>> 
>> Is there perhaps a third option that was absent from the original post?
>> e.g. organizing tests in a separate, optional, potentially
>> pip-installable package.
>> 
>> 
>> >
>> > > outside the module like this:
>> > >
>> > >  https://github.com/pypa/sampleproject/tree/master/tests
>> >
>> > There is no actual reason to do that except win a couple kilobytes if
>> > you are distributing your package on floppy disks for consumption on
>> > Z80-based machines with 64KB RAM.
>> >
>> > Even Python *itself* puts its test suite inside the standard library,
>> > not outside it (though some Linux distros may strip it away).
>> > Try "python -m test.regrtest" (again, this may fail if your distro
>> > decided to ship the test suite in a separate package).
>> >
>> > The PyP"A" should definitely fix its sample project to reflect good
>> > practices.
>> >
>> > Regards
>> >
>> > Antoine.
>> >
>> >
>> > _______________________________________________
>> > Distutils-SIG maillist  -  Distutils-SIG at python.org
>> > https://mail.python.org/mailman/listinfo/distutils-sig
>> _______________________________________________
>> Distutils-SIG maillist  -  Distutils-SIG at python.org
>> https://mail.python.org/mailman/listinfo/distutils-sig
> 
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151010/dae7fdcc/attachment-0001.html>