[Python-ideas] unittest: 0 tests pass means failure of the testsuite

[MRAB]

On 2019-03-06 20:12, Matěj Cepl wrote:
> Hi,
> 
> I am a lead maintainer of Python packages in OpenSUSE and I can
> see the pattern of many packagers adding blindly
> 
>      python setup.py test
> 
> to %check section of our SPEC file. The problem is that if the
> package doesn't use unittest (it actually uses nose, pytest or
> something), it could lead to zero found tests, which pass and
> Python returns exit code 0 (success) even though nothing has been
> tested. It seems from the outside that everything is all right,
> package is being tested on every build, but actually it is lie.
> 
> Would it be possible to change unittest runner, so that when 0
> tests pass, whole test suite would end up failing?
> 
> Thank you for considering this,
> 
Strictly speaking, it's not a lie because none of the tests have failed.