Pytest fails with the wrong error when unicode_literals is present
TL;DR; The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame. Long explanation Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors. The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame To better illustrate what happens, see this gist <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a> The tests <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-test_unicode...> have the following output: - test_plugin_fixture - *fail* - test_conftest_fixture - *fail* - test_inline_fixture - pass - test_no_fixture - pass The output of *test_plugin_fixture* (omitted some output for brevity): $ pytest -k test_plugin_fixture msg = e.value.msg
assert msg.startswith('assert <classes.ValueAndString object'), msg
E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error! While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib <https://bitbucket.org/hpk42/py> library. The “wrong” behaviour can be seen on the run_pyframe_eval.py <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-run_pyframe_...> . I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.
Hi everyone, Besides the problem that reinterpret is executing code in a context without unicode_literals enabled, this also showcases that when moving a fixture with has asserts on it from a test file to a conftest file, asserts will not be rewritten anymore and reinterpretation will be used instead, which might cause unexpected issues, as illustrated by Gustavo. The first issue looks like a bug to me, not sure what to do about this second point. Any thoughts? Cheers, On Mon, Nov 24, 2014 at 1:51 PM, Edison Gustavo Muenz < edisongustavo@gmail.com> wrote:
TL;DR;
The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame. Long explanation
Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors.
The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame
To better illustrate what happens, see this gist <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a>
The tests <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-test_unicode...> have the following output:
- test_plugin_fixture - *fail* - test_conftest_fixture - *fail* - test_inline_fixture - pass - test_no_fixture - pass
The output of *test_plugin_fixture* (omitted some output for brevity):
$ pytest -k test_plugin_fixture
msg = e.value.msg
assert msg.startswith('assert <classes.ValueAndString object'), msg
E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error!
While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib <https://bitbucket.org/hpk42/py> library. The “wrong” behaviour can be seen on the run_pyframe_eval.py <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-run_pyframe_...> .
I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Hi, So if I understand all this correctly there are two things going on: 1) There's a bug with re-interpret which does not respect the unicode_literals from the original source. 2) re-interpret is being used instead of rewrite because an assert occurs in a file which is not a test module itself. The second here is basically the same as https://bitbucket.org/hpk42/pytest/issue/612/syntax-error-on-assertion-reint... IIUC. To be perfectly honest my personal reaction to this is and has always been (before I was aware that this is a way to trigger re-interpret) that one should not have assertions in fixtures. Now experience seems to indicate I might be rather lonely in that opinion so maybe that's not a too useful stance. I've been trying to think about this a few times but haven't really found a useful answer to this problem yet. As for the former issue here: I'm not sure if anyone is going to have a lot of enthusiasm to fix this given that I think we would kind of like to drop re-interpret at some point. I suspect some features have been sneaking in which do not properly work in re-interpret by now and it's generally not been getting much love since rewrite came about. So I think it's much more important to try and find a solution for problem of re-write being avoided. But obviously a pull request with a fix would still be gratefully accepted. Regards, Floris On 25 November 2014 at 12:49, Bruno Oliveira <nicoddemus@gmail.com> wrote:
Hi everyone,
Besides the problem that reinterpret is executing code in a context without unicode_literals enabled, this also showcases that when moving a fixture with has asserts on it from a test file to a conftest file, asserts will not be rewritten anymore and reinterpretation will be used instead, which might cause unexpected issues, as illustrated by Gustavo.
The first issue looks like a bug to me, not sure what to do about this second point. Any thoughts?
Cheers,
On Mon, Nov 24, 2014 at 1:51 PM, Edison Gustavo Muenz <edisongustavo@gmail.com> wrote:
TL;DR;
The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame.
Long explanation
Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors.
The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame
To better illustrate what happens, see this gist
The tests have the following output:
test_plugin_fixture - fail test_conftest_fixture - fail test_inline_fixture - pass test_no_fixture - pass
The output of test_plugin_fixture (omitted some output for brevity):
$ pytest -k test_plugin_fixture
msg = e.value.msg
assert msg.startswith('assert <classes.ValueAndString object'), msg
E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error!
While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib library. The “wrong” behaviour can be seen on the run_pyframe_eval.py.
I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Hi Floris, On Thu, Dec 4, 2014 at 7:14 PM, Floris Bruynooghe <flub@devork.be> wrote:
1) There's a bug with re-interpret which does not respect the unicode_literals from the original source.
2) re-interpret is being used instead of rewrite because an assert occurs in a file which is not a test module itself.
The second here is basically the same as
https://bitbucket.org/hpk42/pytest/issue/612/syntax-error-on-assertion-reint... IIUC.
Exactly. :) To be perfectly honest my personal reaction to this is and has always
been (before I was aware that this is a way to trigger re-interpret) that one should not have assertions in fixtures. Now experience seems to indicate I might be rather lonely in that opinion so maybe that's not a too useful stance. I've been trying to think about this a few times but haven't really found a useful answer to this problem yet.
Since pytest knows all fixtures, and indirectly their source files, perhaps a solution would be to also rewrite asserts in all modules where fixtures are defined.
As for the former issue here: I'm not sure if anyone is going to have a lot of enthusiasm to fix this given that I think we would kind of like to drop re-interpret at some point. I suspect some features have been sneaking in which do not properly work in re-interpret by now and it's generally not been getting much love since rewrite came about. So I think it's much more important to try and find a solution for problem of re-write being avoided. But obviously a pull request with a fix would still be gratefully accepted.
It seems doable: Frame uses eval() to re-evaluate the failed assert expression; it would have to be changed use compile() passing the appropriate flags, which can be obtained from the original code object from the frame with the failed assert. Cheers, Bruno On 25 November 2014 at 12:49, Bruno Oliveira <nicoddemus@gmail.com> wrote:
Hi everyone,
Besides the problem that reinterpret is executing code in a context without unicode_literals enabled, this also showcases that when moving a fixture with has asserts on it from a test file to a conftest file, asserts will not be rewritten anymore and reinterpretation will be used instead, which might cause unexpected issues, as illustrated by Gustavo.
The first issue looks like a bug to me, not sure what to do about this second point. Any thoughts?
Cheers,
On Mon, Nov 24, 2014 at 1:51 PM, Edison Gustavo Muenz <edisongustavo@gmail.com> wrote:
TL;DR;
The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame.
Long explanation
Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails
I get
the wrong kind of errors.
The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame
To better illustrate what happens, see this gist
The tests have the following output:
test_plugin_fixture - fail test_conftest_fixture - fail test_inline_fixture - pass test_no_fixture - pass
The output of test_plugin_fixture (omitted some output for brevity):
$ pytest -k test_plugin_fixture
msg = e.value.msg
assert msg.startswith('assert <classes.ValueAndString object'), msg
E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error!
While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib library. The “wrong” behaviour can be seen on the run_pyframe_eval.py.
I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
_______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
Floris, you are correct. Those are the 2 issues. As for the issue number *1* I was thinking that this could be solved in the *pylib* <https://bitbucket.org/hpk42/py> library. As for the solution I don’t know if the imports should be parsed out from the fullsource attribute as that might be slow. If that’s the solution I could create a pull-request. Regards, Gustavo
participants (3)
-
Bruno Oliveira -
Edison Gustavo Muenz -
Floris Bruynooghe