Re: [pytest-dev] how to fail on exceptions in other threads and destructors
On 06/01/2013 06:42 PM, Adam Goucher wrote:
I would like to make sure that a test fails when it results in an uncaught exception - even if the exception happens in a destructor or in a separate thread.
What's the best way to do this?
My idea is to modify sys.excepthook such that it keeps a list of all the exceptions encountered during a test run. Then, at the end of every test, I could check this list and raise an exception if there were any calls to sys.excepthook.
However, I'm not quite sure how to do this properly without having to request an explicit fixture in every test. Is there a way to have initialization and finalization code run for every test function? And what kind of exception would I need to raise in the finalization code for py.test to recognize that it is the test that failed (rather than a bug in the finalization code)?
IIRC, I think what you want to do is look at the pytest_runtest_makereport hook. It gets called at each part of the execution flow. You would then look at call.when to determine when the thing that went wrong, went wrong.
https://github.com/Element-34/Py.Saunter-Examples/blob/master/picpaste/webdr... is kinda an example. Though in my case I ignore the setup and teardown bits.
That looks close to what I want to do, thanks for the link! However, I have some trouble understanding what's actually happening here: Am I correct that, despite its name, the pytest_runtest_makereport hook is called not just after the test has been teared down, but also after setup and calling? Or is it called before setup, calling and teardown? Also, what exactly is the __multicall__ argument? According to http://pytest.org/latest/plugins.html#py-test-hook-reference, pytest_runtest_makereport accepts just two (item, call) parameters...? Best, Nikolaus
On Mon, Jun 03, 2013 at 09:32 -0700, Nikolaus Rath wrote:
On 06/01/2013 06:42 PM, Adam Goucher wrote:
I would like to make sure that a test fails when it results in an uncaught exception - even if the exception happens in a destructor or in a separate thread.
What's the best way to do this?
My idea is to modify sys.excepthook such that it keeps a list of all the exceptions encountered during a test run. Then, at the end of every test, I could check this list and raise an exception if there were any calls to sys.excepthook.
However, I'm not quite sure how to do this properly without having to request an explicit fixture in every test. Is there a way to have initialization and finalization code run for every test function? And what kind of exception would I need to raise in the finalization code for py.test to recognize that it is the test that failed (rather than a bug in the finalization code)?
IIRC, I think what you want to do is look at the pytest_runtest_makereport hook. It gets called at each part of the execution flow. You would then look at call.when to determine when the thing that went wrong, went wrong.
https://github.com/Element-34/Py.Saunter-Examples/blob/master/picpaste/webdr... is kinda an example. Though in my case I ignore the setup and teardown bits.
That looks close to what I want to do, thanks for the link!
However, I have some trouble understanding what's actually happening here:
Am I correct that, despite its name, the pytest_runtest_makereport hook is called not just after the test has been teared down, but also after setup and calling? Or is it called before setup, calling and teardown?
yes, pytest_runtest_makereport is called for each of the three setup/call/teardown phases of running a test. The phase is indicated in ``call.when``.
Also, what exactly is the __multicall__ argument? According to http://pytest.org/latest/plugins.html#py-test-hook-reference, pytest_runtest_makereport accepts just two (item, call) parameters...?
"__multicall__" is missing docs because it wasn't yet part of the official API. But i consider it now part of the official API FWIW :) Any hook implementation can accept ``__multicall__`` in addition to the hook-specified parameters. ``__multicall__.execute()`` allows to call the pending hook implementations, thus allowing to wrap values. Concretely, the following pattern allows to decorate/modify values from other hooks: @pytest.mark.tryfirst # tell pytest to run this hook impl early def pytest_SOMEHOOK(..., __multicall__): result = __multicall__.execute() # execute all other hook impls # decorate/process result return result cheers, holger
Best, Nikolaus _______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
participants (2)
-
holger krekel -
Nikolaus Rath