[Distutils] Where should I put tests when packaging python modules?

[Wes Turner]

On Oct 7, 2015 6:58 AM, "Ionel Cristian Mărieș" <contact at ionelmc.ro> wrote:
>
>
> On Wed, Oct 7, 2015 at 8:12 AM, Thomas Güttler <
guettliml at thomas-guettler.de> wrote:
>>
>> I thought "easy_install" is a very old and deprecated method.
>
>
> ​Indeed it is. That why people put all sorts of custom "test" commands in
their setup.py to work around the deficiencies of the "test​
> ​" command setuptools provides.​ So we end up with lots of variations of
"how to use pytest to run tests via `setup.py test`", "how to use pip to
install deps, instead of what `setup.py test` normally does" and so on.
>
> If you're gonna implement a test runner in your setup.py you might as
well use a supported and well maintained tool: tox.
>
>>
>> Why not use `setup.py test`?
>
>
> ​Because:
>
> 1. There's Tox​, which does exactly that, and more. It's maintained. It
gets features.

Tox rocks.

* detox can run concurrent processes:
   https://pypi.python.org/pypi/detox/

* TIL timeit.default_timer measures **wall time** by default and not CPU
time: concurrent test timings are likely different from linear tests run on
an machine with load

> 2. The "test" command will install the "test_requires" dependencies as
eggs. You will end up with multiple versions of the same eggs right in your
source checkout.

* is there no way around this?
* is this required / spec'd / fixable?

> 3. The "test" command will install the "test_requires" dependencies with
easy_install. That means wheels cannot be used.

would it be possible to add this to wheel?

as if, after package deployment, in-situ tests are no longer relevant.

(I think it wise to encourage TDD here)

> 4. Because the builtin "test" command is so bare people tend to implement
a custom one. Everyone does something slightly different, and slightly
buggy.

* README.rst test invocation examples (all, subset, one)
* Makefile (make test; [vim] :make)
* python setup.py nosetests
   http://nose.readthedocs.org/en/latest/api/commands.html
* python setup.py [test]

https://pytest.org/latest/goodpractises.html#integrating-with-setuptools-python-setup-py-test

> 5. There's no established tooling that relies on `setup.py test`. There
isn't even a test result protocol like TAP [1] for it. Why use something so
limited and outdated if there's no practical advantage?

* xUnit XML:
   https://westurner.org/wiki/awesome-python-testing#xunit-xml
```
xUnit XML⬅
https://en.wikipedia.org/wiki/XUnit

https://nose.readthedocs.org/en/latest/plugins/xunit.html
http://nosexunit.sourceforge.net/
https://pytest.org/latest/usage.html#creating-junitxml-format-files
https://github.com/xmlrunner/unittest-xml-reporting
https://github.com/zandev/shunit2/compare/master...jeremycarroll:master
```
* TAP protocol

>
> [1] https://testanything.org/
>
> Thanks,
> -- Ionel Cristian Mărieș, http://blog.ionelmc.ro
>
> _______________________________________________
> Distutils-SIG maillist  -  Distutils-SIG at python.org
> https://mail.python.org/mailman/listinfo/distutils-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20151007/e005de40/attachment-0001.html>