[pytest-dev] how to fail on exceptions in other threads and destructors

[Nikolaus Rath]

On 06/01/2013 06:42 PM, Adam Goucher wrote:
>> I would like to make sure that a test fails when it results in an
>> uncaught exception - even if the exception happens in a destructor or in
>> a separate thread.
>>
>> What's the best way to do this?
>>
>> My idea is to modify sys.excepthook such that it keeps a list of all the
>> exceptions encountered during a test run. Then, at the end of every
>> test, I could check this list and raise an exception if there were any
>> calls to sys.excepthook.
>>
>> However, I'm not quite sure how to do this properly without having to
>> request an explicit fixture in every test. Is there a way to have
>> initialization and finalization code run for every test function? And
>> what kind of exception would I need to raise in the finalization code
>> for py.test to recognize that it is the test that failed (rather than a
>> bug in the finalization code)?
>
> IIRC, I think what you want to do is look at the pytest_runtest_makereport hook. It gets called at each part of the execution flow. You would then look at call.when to determine when the thing that went wrong, went wrong.
> 
> https://github.com/Element-34/Py.Saunter-Examples/blob/master/picpaste/webdriver/conftest.py#L34 is kinda an example. Though in my case I ignore the setup and teardown bits.


That looks close to what I want to do, thanks for the link!

However, I have some trouble understanding what's actually happening here:

Am I correct that, despite its name, the pytest_runtest_makereport hook
is called not just after the test has been teared down, but also after
setup and calling? Or is it called before setup, calling and teardown?

Also, what exactly is the __multicall__ argument? According to
http://pytest.org/latest/plugins.html#py-test-hook-reference,
pytest_runtest_makereport accepts just two (item, call) parameters...?


Best,
Nikolaus