[pytest-dev] pytest_runtest_call hook skipped in strange manner

[holger krekel]

Hi Anton,

pytest_runtest_call is responsible for executing the test.  An exception
will escalate, so not all plugin hook impls might execute in the case of
a failing test.

If you use @pytest.mark.tryfirst before the pytest_runtest_call you should
be able to avoid the problem - your hook will run.

HTH,
holger



On Wed, Feb 06, 2013 at 13:56 +0200, Anton P wrote:
> Sorry for confusing, but it looks like runtest_call hook defined in plugin
> is always skiped in case test is failed.
> I've tried to add there some output to file to be sure that there is no
> issue with pytest stdout capturing.
> This issue creates a problem for me. Because plugin does some stuff before
> test and try to verify that stuff is good after test finished. But if tc
> could be failed (does py.test can predict the result?!) runtest_call hook
> is skipped but runtest_teardown hook is run as usual. Therefore I get
> double error: tc failed, teardown failed.
> 
> If nobody has time to check this, than please point me where I can check
> the issue by myself. Why the hook could be skipped?
> 
> Ubuntu Linux 12.04 -- Python 2.7.3 -- pytest-2.3.4
> 
> Thank you in advance!
> -Anton
> 
> 
> On Tue, Feb 5, 2013 at 4:51 PM, Anton P <anton7811 at gmail.com> wrote:
> 
> > Hi All,
> >
> > I'm faced with very strange issue with runtest_call hook.
> > I've create plugin with such hook and it's working good unless you have
> > the following code in the test case:
> > assert env.switch[1].type == "real"
> > If you change at least on sign in this assertion (e.g. != "real" or ==
> > "unreal") the issue cannot be reproduced and hook is performed correctly.
> >
> > Could anybody explain me what is going with this test case?
> >
> > Thank you in advance!
> > -Anton
> >
> >
> > Here the code:
> >
> >
> > $ tree
> > .
> > ├── conftest.py
> > ├── pytest_fake.py
> > └── test_temp.py
> >
> > *1. conftest.py*
> > #! /usr/bin/env python
> >
> > pytest_plugins = ["pytest_fake", ]
> >
> > def pytest_runtest_call(item):
> >     print "\nconftest runtest call"
> >
> > def pytest_runtest_teardown(item, nextitem):
> >     print "\nconftest runtest teardown"
> >
> > *2. pytest_fake.py*
> > #!/usr/bin/env python
> >
> > import pytest
> >
> >
> > def pytest_addoption(parser):
> >     group = parser.getgroup("fake", "plugin fake")
> >     group.addoption("--fake", action="store_true", dest="fake",
> >                     default=False,
> >                     help="Enable fake plugin. %default by default.")
> >
> >
> > def pytest_configure(config):
> >     if config.option.fake:
> >         config.pluginmanager.register(Fake(config.option), "_fake")
> >
> >
> > def pytest_unconfigure(config):
> >     fake = getattr(config, "_fake", None)
> >     if fake:
> >         del config._fake
> >         config.pluginmanager.unregister(fake)
> >
> >
> > class Fake(object):
> >
> >     def __init__(self, opts):
> >         pass
> >
> >     @pytest.mark.trylast
> >     def pytest_runtest_call(self, item):
> >         print "runtest_call - OK"
> >
> >     @pytest.mark.tryfirst
> >     def pytest_runtest_teardown(self, item, nextitem):
> >         print "runtest_teardown"
> >
> > *3. test_temp.py*
> > #! /usr/bin/env python
> >
> > class A:
> >     pass
> >
> > class B:
> >     def __init__(self):
> >         self.switch = {}
> >         self.switch[1] = A()
> >         self.switch[1].type = "unreal"
> >
> > class TestTemp(object):
> >     def test_tmp_01(self):
> >         var = B()
> >         assert var.switch[1].type == "real"
> >
> >     def test_tmp_02(self):
> >         env1 = B()
> >         assert env1.switch[1].type != "real"
> >
> >
> >
> > *Execution log:*
> >
> > I'm expecting that message "runtest_call - OK" will appear before the
> > first case.
> >
> > $ py.test -s -v --fake test_temp.py
> > ================================================= test session starts
> > =================================================
> > platform linux2 -- Python 2.7.3 -- pytest-2.3.4 -- /usr/bin/python
> > collected 2 items
> >
> > test_temp.py:13: TestTemp.test_tmp_01
> > conftest runtest call
> > FAILEDruntest_teardown
> >
> > conftest runtest teardown
> >
> > test_temp.py:17: TestTemp.test_tmp_02
> > conftest runtest call
> > runtest_call - OK
> > PASSEDruntest_teardown
> >
> > conftest runtest teardown
> >
> >
> > ====================================================== FAILURES
> > =======================================================
> > ________________________________________________ TestTemp.test_tmp_01
> > _________________________________________________
> >
> > self = <test_temp.TestTemp object at 0x15e6f90>
> >
> >     def test_tmp_01(self):
> >         var = B()
> > >       assert var.switch[1].type == "real"
> > E       assert 'unreal' == 'real'
> > E         - unreal
> > E         ? --
> > E         + real
> >
> > self       = <test_temp.TestTemp object at 0x15e6f90>
> > var        = <test_temp.B instance at 0x16293f8>
> >
> > test_temp.py:15: AssertionError
> > =============================================== slowest test durations
> > ================================================
> > 0.00s call     test_temp.py::TestTemp::test_tmp_01
> > 0.00s setup    test_temp.py::TestTemp::test_tmp_01
> > 0.00s teardown test_temp.py::TestTemp::test_tmp_01
> > 0.00s call     test_temp.py::TestTemp::test_tmp_02
> > 0.00s setup    test_temp.py::TestTemp::test_tmp_02
> > 0.00s teardown test_temp.py::TestTemp::test_tmp_02
> > ========================================= 1 failed, 1 passed in 0.02
> > seconds ==========================================
> >
> >

> _______________________________________________
> Pytest-dev mailing list
> Pytest-dev at python.org
> http://mail.python.org/mailman/listinfo/pytest-dev