[pytest-dev] Custom reporting for asserts without comparison operators?

[Shawn Brown]

Would a return value helper--as you are thinking about it--be able to
handle cases like test_3passing()?

    def test_3passing():
        with pytest.raises(AssertionError) as excinfo:
            assert myfunc(41)
        assert 'custom report' in str(excinfo.value)

I've looked around the code base but I'm not sure where best to hack on
things. I've played with changes to _pytest.assertion.rewrite._saferepr()
but this doesn't seem to be the right place to address this sort of change
(newline handling and other details are handled elsewhere).


On Thu, Mar 22, 2018 at 4:12 AM, Ronny Pfannschmidt <rpfannsc at redhat.com>
wrote:

> that approach is broken in the sense, that it breaks behaviour
> expectations,
> an return value helper, that triggers an assertion on its own is simply no
> longer a return value helper, but a assertion helper
>
> supporting it like that would result in a really bad api
>
> instead having  assertion helper that returns a "truthy" object which can
> be introspected by pytest and/or negated should be more suitable
>
> 2018-03-22 3:39 GMT+01:00 Shawn Brown <03sjbrown at gmail.com>:
>
>> Ah. It's good to see that this has been thought about before.
>>
>> My motivation for asking this question was to perform my due diligence
>> and make sure I wasn't missing something before moving ahead. My
>> immediate need is handled by using assert_myfunc() to raise its own error
>> internally--same as Floris' example. Though, it's not ideal.
>>
>> I know my examples have been vague as I've stripped the specifics of my
>> project to focus the question specifically on pytest's behavior and I
>> greatly appreciate everyone who is giving some thought to this.
>>
>> As Ronny mentioned, I'm sure it's possible to address this without
>> user-facing AST manipulation. But I'm not familiar enough with the code
>> base to see where I can best hack on the representations. However, I do
>> have a working AST-based demonstration (below). This uses a fragile
>> monkey-patch that is just asking for trouble so please take this for the
>> experimental hack it is...
>>
>>
>> FILE "conftest.py":
>>
>>     import ast
>>     import _pytest
>>
>>     def my_ast_prerewrite_hook(ast_assert):
>>         """Modifies AST of certain asserts before pytest-rewriting."""
>>         # Demo AST-tree manipulation (actual implemenation
>>         # would need to be more careful than this).
>>         if (isinstance(ast_assert.test, ast.Call)
>>                 and isinstance(ast_assert.test.func, ast.Name)
>>                 and ast_assert.test.func.id == 'myfunc'):
>>
>>             ast_assert.test.func = ast.Name('assert_myfunc', ast.Load())
>>
>>         return ast_assert
>>
>>     # UNDESIRABLE MONKEY PATCHING!!!
>>     class ModifiedRewriter(_pytest.assertion.rewrite.AssertionRewriter):
>>         def visit_Assert(self, assert_):
>>             assert_ = my_ast_prerewrite_hook(assert_)  # <- PRE-REWRITE
>> HOOK
>>             return super(ModifiedRewriter, self).visit_Assert(assert_)
>>
>>     def rewrite_asserts(mod, module_path=None, config=None):
>>         ModifiedRewriter(module_path, config).run(mod)
>>
>>     _pytest.assertion.rewrite.rewrite_asserts = rewrite_asserts
>>
>>
>> FILE "test_ast_hook_approach.py":
>>
>>     import pytest
>>
>>     # Test helpers.
>>     def myfunc(x):
>>         return x == 42
>>
>>     def assert_myfunc(x):
>>         __tracebackhide__ = True
>>         if not myfunc(x):
>>             msg = 'custom report\nmulti-line output\nmyfunc({0}) failed'
>>             raise AssertionError(msg.format(x))
>>         return True
>>
>>     # Test cases.
>>     def test_1passing():
>>         assert myfunc(42)
>>
>>     def test_2passing():
>>         assert myfunc(41) is False
>>
>>     def test_3passing():
>>         with pytest.raises(AssertionError) as excinfo:
>>             assert myfunc(41)
>>         assert 'custom report' in str(excinfo.value)
>>
>>     def test_4failing():
>>         assert myfunc(41)
>>
>>
>> Running the above test gives 3 passing cases and 1 failing case (which
>> uses the custom report). Also, test_2passing() checks for "is False"
>> instead of just "== False" which I think would be wonderful to support as
>> it removes all caveats for the user (so users get a real False when they
>> expect False, instead of a Falsey alternative). Also, if I were going to
>> use AST manipulation like this, I would probably reference assert_myfunc()
>> by attaching it as a private attribute to myfunc() itself -- and then
>> reference it with ast.Attribute() node instead of an ast.Name(). But again,
>> solving this without AST manipulation could be better in many ways.
>>
>> --Shawn
>>
>>
>> On Mon, Mar 19, 2018 at 1:59 PM, Ronny Pfannschmidt <
>> ich at ronnypfannschmidt.de> wrote:
>>
>>> hi everyone,
>>>
>>> this is just about single value assertion helpers
>>>
>>> i logged an feature request about that a few year back
>>> see https://github.com/pytest-dev/pytest/issues/95 -
>>>
>>> so basically this use-case was known since 2011 ^^ and doesn't require
>>> ast rewriting lice macros,
>>> just proper engineering of the representation and handling of single
>>> values in the assertion rewriter.
>>>
>>> -- Ronny
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20180322/b1741b2f/attachment-0001.html>