Mailman 3 December 2014 - pytest-dev

Crowdfunding a pytest plugin
by Tibor Arpas March 13, 2015

March 13, 2015

Hello pytesters, We are putting together a small Indiegogo campaign for a pytest plugin which should significantly improve the experience of executing a long test suite. We'd be glad to hear any feedback before launching it. The draft is here: https://www.indiegogo.com/project/preview/4fe07d66 Also, is anybody interested in helping with promotion and/or development? Cheers, Tibor

4 6

Enabling viewing of rewritten AST trees as Python
by Tom Viner March 2, 2015

March 2, 2015

Hi, In the process of fixing a recent bug <https://bitbucket.org/hpk42/pytest/issue/615/valueerror-on-compound-assert-…> in pytest source rewriting I found a library to take the rewritten AST tree and generate Python source code from it. Floris mentioned this could be helpful to pytest developers working on the assertion rewriting code. The library is called Meta <https://pypi.python.org/pypi/meta>. (Note, currently only the develop branch on github works successfully on pytest's rewrite tests.) It allows viewing an AST tree as Python source with a single call: import meta log.debug(meta.asttools.dump_python_source(ast_tree)) The question I have: how can pytest developers (and those debugging pytest's rewriting of their tests) best get access to the rewritten AST tree, to be able to view it's Python source representation? The AST tree is seen in pytest/_pytest/assertion/rewrite.py::rewrite_asserts <https://bitbucket.org/hpk42/pytest/src/default/_pytest/assertion/rewrite.py…> in the mod variable: def rewrite_asserts(mod): """Rewrite the assert statements in mod.""" AssertionRewriter().run(mod) The four options I can think of to gain access to this variable: - monkey patch rewrite.rewrite_asserts - edit rewrite_asserts and paste in the logging code above: -- temporarily, by each developer, on each occasion required -- permanently, but only running when a certain debug flag is active - expose the rewritten AST via a new hook monkey patch is my preferred option, because if I can make it work, a plugin could be written. I just need some help to find an appropriate hook to do the monkey patching before pytest gets going with the AssertionRewritingHook and calls rewrite_asserts. Without knowledge of an early-running hook, I could only get this working by patching rewrite_asserts then calling pytest.main myself, see https://gist.github.com/tomviner/13c95cdb1e159028fc0b Second option, I could explain how to do the temporary edit in a short blog post. This would get past the pytest.main downside to the gist above. But telling people to edit random pytest source files seems bad form. Third option, a pull request on pytest logging the Python source, only if a debug flag is active and Meta is importable. But is this reasonable just for debugging purposes? Alternatively, fourth, a new hook could be added to expose the rewritten AST tree, and everything else could then be a plugin. Thoughts? To see the assertion rewriting pytest does, as Python code, follow the instructions in my gist: https://gist.github.com/tomviner/13c95cdb1e159028fc0b Cheers, Tom

2 3

pytest github organization is taken
by Anatoly Bubenkov Jan. 29, 2015

Jan. 29, 2015

Hi all if it was you who created this org https://github.com/pytest please make Holger an admin if this name was taken by someone who's not related to pytest in any way, then we should make a claim to github to get the rights i guess the org is empty and created Oct 21, 2013 -- Anatoly Bubenkov

3 5

Exemplary py.test usage in open-source?
by Hashemi, Mahmoud Dec. 25, 2014

Dec. 25, 2014

Hey pytest-dev, Had a casual question for you all. We all know py.test is a great testing framework with a lot of top-notch documentation and plugins; several teams here at eBay/PayPal make good use of it (some of which is viewable on eBay's public Github). But some folks around here learn best from example. So in the spirit of open-source, would anyone mind sharing some Github/Bitbucket links to codebases that leverage py.test to its fullest? I'm especially interested in larger, more "enterprise" codebases with multifaceted tests. Werkzeug, devpi, and other infrastructural code is great, please share your faves along those lines, but I'm thinking more in the vein of Mozilla's tests (https://github.com/mozilla?query=tests). Thanks! Mahmoud Lead Developer, Python Infrastructure at eBay/PayPal https://github.com/mahmoud

3 4

Running a parametrized test case in multiple CPUs
by Goncalo Morgado Dec. 8, 2014

Dec. 8, 2014

Hi pytest community, I am not sure this is the right place to ask this, and I am sorry if it's not. I am trying to make use of multiple CPUs to run my tests, which I can for separately defined tests (different test_ functions for instance). *The challenge is to run tests, generated from one single test_ function which fixture it depends on is parametrized with multiple parameters, on multiple CPUs*. Currently I have this test that generates more than a thousand tests (because of the parametrized fixture it depends on) and ends up being very time consuming to run in one single CPU. Thanks for this great tool that is pytest! Cheers, Gonçalo

2 8

Pytest fails with the wrong error when unicode_literals is present
by Edison Gustavo Muenz Dec. 8, 2014

Dec. 8, 2014

TL;DR; The eval() method of the Frame object is not “inheriting” the __future__ imports from the frame. Long explanation Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors. The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame To better illustrate what happens, see this gist <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a> The tests <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-test_unicod…> have the following output: - test_plugin_fixture - *fail* - test_conftest_fixture - *fail* - test_inline_fixture - pass - test_no_fixture - pass The output of *test_plugin_fixture* (omitted some output for brevity): $ pytest -k test_plugin_fixture msg = e.value.msg > assert msg.startswith('assert <classes.ValueAndString object'), msg E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error! While debugging I found out that the ValueAndString constructor was being called 3 times, instead of 2. The 3rd call happened in code.py:100, which is part of the py lib <https://bitbucket.org/hpk42/py> library. The “wrong” behaviour can be seen on the run_pyframe_eval.py <https://gist.github.com/edisongustavo/5533a1c36de6cab6f39a#file-run_pyframe…> . I’ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits.

3 4

Access markers from pytest_generate_tests?
by Nikolaus Rath Dec. 1, 2014

Dec. 1, 2014

Hello, Is there a way to access test markers sin the pytest_generate_tests hook? For example: @pytest.mark.bla def test_one(): # ... def pytest_generate_tests(metafunc): if metafunc_has_bla_mark(metafunc): # do something But what do you have to use for metafunc_has_bla_mark? Unfortunately metafunc.function does not have a `keywords` attribute... Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

2 1

Help with parametrized fixtures
by Nikolaus Rath Dec. 1, 2014

Dec. 1, 2014

Hello, I have a parametrized fixture foo, and some test cases that use it: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... return foo_f def test_1(foo): # ... def test_2(foo): # .... Now, I'd like to add an additional test, but running it only makes sense for some of the fixture parameters. So far I've solved this by providing the parameter value as a fixture attribute and skipping "unsupported" values: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... foo_f.param = request.param return foo_f def test_3(foo): if foo.param not in (1,4,5): raise pytest.skip("doesn't make sense") This works, but I don't like it very much because it means that the test suite can never be executed without skipping some tests. I'd rather reserve skipping for cases where the test could in principle be executed, but for some reason cannot work at the moment (e.g. because there's no internet, or a required utility is not installed). Is there a way to solve the above scenario without skipping tests? Ideally, I'd be able to do something like this: @only_for_param((1,4,5)) def test_3(foo): ... Thanks! -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

3 3