Mailman 3 November 2014 - pytest-dev

Enabling viewing of rewritten AST trees as Python
by Tom Viner March 2, 2015

March 2, 2015

Hi, In the process of fixing a recent bug <https://bitbucket.org/hpk42/pytest/issue/615/valueerror-on-compound-assert-…> in pytest source rewriting I found a library to take the rewritten AST tree and generate Python source code from it. Floris mentioned this could be helpful to pytest developers working on the assertion rewriting code. The library is called Meta <https://pypi.python.org/pypi/meta>. (Note, currently only the develop branch on github works successfully on pytest's rewrite tests.) It allows viewing an AST tree as Python source with a single call: import meta log.debug(meta.asttools.dump_python_source(ast_tree)) The question I have: how can pytest developers (and those debugging pytest's rewriting of their tests) best get access to the rewritten AST tree, to be able to view it's Python source representation? The AST tree is seen in pytest/_pytest/assertion/rewrite.py::rewrite_asserts <https://bitbucket.org/hpk42/pytest/src/default/_pytest/assertion/rewrite.py…> in the mod variable: def rewrite_asserts(mod): """Rewrite the assert statements in mod.""" AssertionRewriter().run(mod) The four options I can think of to gain access to this variable: - monkey patch rewrite.rewrite_asserts - edit rewrite_asserts and paste in the logging code above: -- temporarily, by each developer, on each occasion required -- permanently, but only running when a certain debug flag is active - expose the rewritten AST via a new hook monkey patch is my preferred option, because if I can make it work, a plugin could be written. I just need some help to find an appropriate hook to do the monkey patching before pytest gets going with the AssertionRewritingHook and calls rewrite_asserts. Without knowledge of an early-running hook, I could only get this working by patching rewrite_asserts then calling pytest.main myself, see https://gist.github.com/tomviner/13c95cdb1e159028fc0b Second option, I could explain how to do the temporary edit in a short blog post. This would get past the pytest.main downside to the gist above. But telling people to edit random pytest source files seems bad form. Third option, a pull request on pytest logging the Python source, only if a debug flag is active and Meta is importable. But is this reasonable just for debugging purposes? Alternatively, fourth, a new hook could be added to expose the rewritten AST tree, and everything else could then be a plugin. Thoughts? To see the assertion rewriting pytest does, as Python code, follow the instructions in my gist: https://gist.github.com/tomviner/13c95cdb1e159028fc0b Cheers, Tom

2 3

pytest github organization is taken
by Anatoly Bubenkov Jan. 29, 2015

Jan. 29, 2015

Hi all if it was you who created this org https://github.com/pytest please make Holger an admin if this name was taken by someone who's not related to pytest in any way, then we should make a claim to github to get the rights i guess the org is empty and created Oct 21, 2013 -- Anatoly Bubenkov

3 5

Exemplary py.test usage in open-source?
by Hashemi, Mahmoud Dec. 25, 2014

Dec. 25, 2014

Hey pytest-dev, Had a casual question for you all. We all know py.test is a great testing framework with a lot of top-notch documentation and plugins; several teams here at eBay/PayPal make good use of it (some of which is viewable on eBay's public Github). But some folks around here learn best from example. So in the spirit of open-source, would anyone mind sharing some Github/Bitbucket links to codebases that leverage py.test to its fullest? I'm especially interested in larger, more "enterprise" codebases with multifaceted tests. Werkzeug, devpi, and other infrastructural code is great, please share your faves along those lines, but I'm thinking more in the vein of Mozilla's tests (https://github.com/mozilla?query=tests). Thanks! Mahmoud Lead Developer, Python Infrastructure at eBay/PayPal https://github.com/mahmoud

3 4

Pytest fails with the wrong error when unicode_literals is present
by Edison Gustavo Muenz Dec. 8, 2014

Dec. 8, 2014

TL;DR; The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame. Long explanation Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors. The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame To better illustrate what happens, see this gist <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a> The tests <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-test_unicod…> have the following output: - test_plugin_fixture - *fail* - test_conftest_fixture - *fail* - test_inline_fixture - pass - test_no_fixture - pass The output of *test_plugin_fixture* (omitted some output for brevity): $ pytest -k test_plugin_fixture msg = e.value.msg > assert msg.startswith('assert <classes.ValueAndString object'), msg E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error! While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib <https://bitbucket.org/hpk42/py> library. The “wrong” behaviour can be seen on the run_pyframe_eval.py <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-run_pyframe…> . I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.

3 4

Access markers from pytest_generate_tests?
by Nikolaus Rath Dec. 1, 2014

Dec. 1, 2014

Hello, Is there a way to access test markers sin the pytest_generate_tests hook? For example: @pytest.mark.bla def test_one(): # ... def pytest_generate_tests(metafunc): if metafunc_has_bla_mark(metafunc): # do something But what do you have to use for metafunc_has_bla_mark? Unfortunately metafunc.function does not have a `keywords` attribute... Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

2 1

Help with parametrized fixtures
by Nikolaus Rath Dec. 1, 2014

Dec. 1, 2014

Hello, I have a parametrized fixture foo, and some test cases that use it: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... return foo_f def test_1(foo): # ... def test_2(foo): # .... Now, I'd like to add an additional test, but running it only makes sense for some of the fixture parameters. So far I've solved this by providing the parameter value as a fixture attribute and skipping "unsupported" values: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... foo_f.param = request.param return foo_f def test_3(foo): if foo.param not in (1,4,5): raise pytest.skip("doesn't make sense") This works, but I don't like it very much because it means that the test suite can never be executed without skipping some tests. I'd rather reserve skipping for cases where the test could in principle be executed, but for some reason cannot work at the moment (e.g. because there's no internet, or a required utility is not installed). Is there a way to solve the above scenario without skipping tests? Ideally, I'd be able to do something like this: @only_for_param((1,4,5)) def test_3(foo): ... Thanks! -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

3 3

pytest_plugin with session scope fixture
by Jens Hoffmann Nov. 25, 2014

Nov. 25, 2014

Hi everybody, I have a problem concering the following setup: - I have written a pytest plugin defining a session scope fixture - in my test files - say ./test/a/test_foo.py and ./test/b/test_bar.py I ``import'' the pytest plugin via declaring the pytest_plugins variable: pytest_plugins = 'my.cool.plugin' So, there's no conftest.py - say below ./test, that declares my.cool.plugin ``globally'' for test/a and test/b What is the session scope in this setup when I run ``py.test test'' on the command line? Is the scope split in test/a and test/b or do we still have a ``real'' session scope for everything below test? My real but similar setup seems to split the scope into chunks of modules which contain test files which declare pytest_plugins... but may be I am wrong... Is that possible? Thank you for your answer, Jens

2 2

yield_fixture still experimental?
by Bruno Oliveira Nov. 16, 2014

Nov. 16, 2014

Hi everyone, The documentation for yield_fixture says: > “yielding” fixture values is an experimental feature and its exact declaration may change later > but earliest in a 2.5 release. You can thus safely use this feature in the 2.4 series but may > need to adapt later. Test functions themselves will not need to change (as a general > feature, they are ignorant of how fixtures are setup). Is yield_fixture still experimental? Are there any plans to change it for 2.7, or later? I ask because a colleague of mind was somewhat cautious about using them because of their "experimental" status. Cheers,

2 2

Extending xdist
by Ivan Kalinin Nov. 14, 2014

Nov. 14, 2014

Hello there, fellow developers. I am now trying to add xdist intergration capabilities for my plugin, pytest-allure-adaptor and I've run into substantial problems with understanding xdist. Efforts so far are discussed here: https://github.com/allure-framework/allure-python/issues/8 . As far as the reverse-enginering leeds me, xdist is running a `master` process and `slave` processes, transmitting pytest's Items to slaves and receiving test results from those. It does so by hand-picking into the pytest hooks, calling some of them on the remote site and some on the master side, effectively achieving the `separated` pytest session. However, pytest-allure-adaptor, being a report-generation-kind plugin, relies significantly on the hook specification and call order, and xdist's hook manipulation renders the allure adaptor incapable of working properly. Here come the questions: 1) Is there any kind of documentation / guide lines / examples of xdist's internal architecture and integration? 2) Are there any planned activity on xdist internals? Or any kind of overhaul? Do xdist developers consider its use of hook an API? Is it sane to rely on xdist internal workings retaining campatibility for at least some time? 3) What's recommended / renown way of developing an xdist-capable pytest plugin? 4) Is it sane to rely on pytest hook call order at all? What is the most other-plugin-friendly way to override / modify pytest hooks? Hope for best and thanks in advance for your time. Best, Ivan.

2 1