[pytest-dev] No traceback filtering with __tracebackhide__ attribute using custom pytest plugin

Ringo De Smet ringo.de.smet at ontoforce.com
Fri Mar 16 04:18:18 EDT 2018


Bruno,

On Thu, Mar 15, 2018 at 11:19 PM, Bruno Oliveira <nicoddemus at gmail.com>
wrote:

> You are right, pytest_runtest_makereport creates a longrepr instance
> depending on excinfo attribute of the call object[1]:
>
> https://github.com/pytest-dev/pytest/blob/fbcf1a90c9ffa849827918249fef17
> 21a1f43bdd/_pytest/runner.py#L294-L307
>
> To do that it calls item.repr_failure(excinfo), which calls
> _repr_failure_py in python.py which ends up calling _prunetraceback which
> filters the traceback here:
>
> https://github.com/pytest-dev/pytest/blob/fbcf1a90c9ffa849827918249fef17
> 21a1f43bdd/_pytest/python.py#L578
>
> .filter() is declared here:
>
> https://github.com/pytest-dev/pytest/blob/fbcf1a90c9ffa849827918249fef17
> 21a1f43bdd/_pytest/_code/code.py#L307
>
> And the default value uses a lambda where ishidden() checks for
> __tracebackhide__ attribute in the function’s locals.
>

You definitely pointed me in the right direction by mentioning that the
filtering passes via an item object. So thanks a lot for this info.

Checking the class hierarchy, I found this (simplified):

class Node(object) # nodes.py
   def _repr_failure_py
   repr_failure = _repr_failure_py
   def _pruntraceback:
       *pass*  # No basic filtering here

class Item(Node) # nodes.py
   ... # no overriden methods here

class Function(FunctionMixin, Item) # python.py
   def _repr_failure_py
   def repr_failure
   def _prunetraceback  # Whole filtering mechanism is triggered here!

And here is my code:

class SpecExample(Item)
   # no overridden methods here

I think it's clear now why it doesn't work. Since I provide instances of
SpecExample to the pytest runner, the filtering defined in FunctionMixin is
not triggered. As a solution I overridden _prunetraceback:

def _prunetraceback(self, excinfo):
    filtered_traceback = excinfo.traceback.filter(filter_traceback)
    filtered_traceback = filtered_traceback.filter()
    excinfo.traceback = filtered_traceback

With this in place, my stack trace becomes this now:

============================= test session starts
==============================
platform darwin -- Python 3.6.4, pytest-3.4.2, py-1.5.2, pluggy-0.6.0
rootdir: /Users/ringods/Projects/ontoforce/metis/execution_layer/spec,
inifile:
plugins: mamba-1.0.0
collected 6 items
action_base_spec.py ....F
action_base_spec.py:None ()
../python-mamba/pytest_mamba/plugin.py:135: in runtest
    raise mamba_error.exception
../../venv/lib/python3.6/site-packages/mamba/example.py:43: in _execute_test
    self.test(execution_context)
action_base_spec.py:44: in 00000007__it is ok--
    expect(rs).to(equal('value2'))
E   AssertionError:
E   expected: 'value' to equal 'value2'
.                                               [100%]

=================================== FAILURES
===================================


Now this is a bit dirty as I have code duplication from what you have in
FunctionMixin. Let's look at pytest from 2 different angles:

   - pytest as a test runner (nodes.py, runner.py)
   - pytest as the test implementation framework (python.py/unittest.py)

I would expect the basic traceback filtering to work via the runner,
irrespective of which custom tests a thirdparty plugin provides. What do I
consider basic filtering? Filtering out the traceback entries of the runner
as defined in python.py:filter_traceback function (pluggy, etc.) as well as
the default filter function regarding __tracebackhide__. This is exactly
what I activated via my custom _prunetraceback method.

May I suggest to push this basic filtering higher in that class hierarchy?
You will probably know better where that is/should be, but I guess in class
Item would be a good fit. Should I file this as an improvement on Github?

Greetings,

Ringo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20180316/4c7919d7/attachment.html>


More information about the pytest-dev mailing list