[pytest-dev] how to fail on exceptions in other threads and destructors
Nikolaus at rath.org
Mon Jun 3 19:30:49 CEST 2013
On 06/03/2013 02:42 AM, holger krekel wrote:
> Hi Nikolaus,
> On Sat, Jun 01, 2013 at 14:41 -0700, Nikolaus Rath wrote:
>> I would like to make sure that a test fails when it results in an
>> uncaught exception - even if the exception happens in a destructor or in
>> a separate thread.
>> What's the best way to do this?
>> My idea is to modify sys.excepthook such that it keeps a list of all the
>> exceptions encountered during a test run. Then, at the end of every
>> test, I could check this list and raise an exception if there were any
>> calls to sys.excepthook.
>> However, I'm not quite sure how to do this properly without having to
>> request an explicit fixture in every test. Is there a way to have
>> initialization and finalization code run for every test function? And
>> what kind of exception would I need to raise in the finalization code
>> for py.test to recognize that it is the test that failed (rather than a
>> bug in the finalization code)?
> You could use an "autouse" fixture which gets called without naming
> the fixture in test/fixture function arguments.
Ah, that's useful. Thanks for the pointer!
> To fail the actual test, you might look into implementing the
> pytest_pyfunc_call() hook. It is responsible for calling a python test
> function. You would want to wrap the existing hook implementation and
> add your exception-detection logic like this:
> # content of conftest.py in your testing dir
> import pytest
> def pytest_pyfunc_call(__multicall__):
> ret = __multicall__.execute() # if this raises we don't care
> # perform extra checks and raise if there is a problem
> return ret # return whatever our wrapped hook impl returns
> The ``__multicall__`` bit is for interacting with other hook
> implementations so that we don't have to replicate their logic.
> You could also use the "pytest_runtest_call" hook if you
> also have doctests or other non-function types of tests.
Could you elaborate on the difference between pytest_pyfunc_call and
pytest_runtest_call? The former doesn't seem to be mentioned anywhere in
the docs. (Some of my tests are unittest.TestCases.)
I'm also not sure about the tryfirst decorator. Why is that needed? Do I
also need it when using pytest_runtest_call?
As a side note, I just tried to set up something based on an autouse
fixture and found that sys.excepthook is ignored both for threads and
destructors. There is a workaround for threads
(http://bugs.python.org/issue1230540), but __del__ seems to be a
hopeless case -- I believe it prints directly to stderr from C code. So
what I'll try instead is to make my extra tests check if a test resulted
in any (or maybe any suspicious) output to stderr.
More information about the Pytest-dev