From nicoddemus at gmail.com Tue Nov 4 13:37:28 2014 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Tue, 4 Nov 2014 10:37:28 -0200 Subject: [pytest-dev] yield_fixture still experimental? Message-ID: Hi everyone, The documentation for yield_fixture says: > ?yielding? fixture values is an experimental feature and its exact declaration may change later > but earliest in a 2.5 release. You can thus safely use this feature in the 2.4 series but may > need to adapt later. Test functions themselves will not need to change (as a general > feature, they are ignorant of how fixtures are setup). Is yield_fixture still experimental? Are there any plans to change it for 2.7, or later? I ask because a colleague of mind was somewhat cautious about using them because of their "experimental" status. Cheers, -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom at viner.tv Sun Nov 9 20:18:33 2014 From: tom at viner.tv (Tom Viner) Date: Sun, 9 Nov 2014 19:18:33 +0000 Subject: [pytest-dev] Enabling viewing of rewritten AST trees as Python Message-ID: Hi, In the process of fixing a recent bug in pytest source rewriting I found a library to take the rewritten AST tree and generate Python source code from it. Floris mentioned this could be helpful to pytest developers working on the assertion rewriting code. The library is called Meta . (Note, currently only the develop branch on github works successfully on pytest's rewrite tests.) It allows viewing an AST tree as Python source with a single call: import meta log.debug(meta.asttools.dump_python_source(ast_tree)) The question I have: how can pytest developers (and those debugging pytest's rewriting of their tests) best get access to the rewritten AST tree, to be able to view it's Python source representation? The AST tree is seen in pytest/_pytest/assertion/rewrite.py::rewrite_asserts in the mod variable: def rewrite_asserts(mod): """Rewrite the assert statements in mod.""" AssertionRewriter().run(mod) The four options I can think of to gain access to this variable: - monkey patch rewrite.rewrite_asserts - edit rewrite_asserts and paste in the logging code above: -- temporarily, by each developer, on each occasion required -- permanently, but only running when a certain debug flag is active - expose the rewritten AST via a new hook monkey patch is my preferred option, because if I can make it work, a plugin could be written. I just need some help to find an appropriate hook to do the monkey patching before pytest gets going with the AssertionRewritingHook and calls rewrite_asserts. Without knowledge of an early-running hook, I could only get this working by patching rewrite_asserts then calling pytest.main myself, see https://gist.github.com/tomviner/13c95cdb1e159028fc0b Second option, I could explain how to do the temporary edit in a short blog post. This would get past the pytest.main downside to the gist above. But telling people to edit random pytest source files seems bad form. Third option, a pull request on pytest logging the Python source, only if a debug flag is active and Meta is importable. But is this reasonable just for debugging purposes? Alternatively, fourth, a new hook could be added to expose the rewritten AST tree, and everything else could then be a plugin. Thoughts? To see the assertion rewriting pytest does, as Python code, follow the instructions in my gist: https://gist.github.com/tomviner/13c95cdb1e159028fc0b Cheers, Tom -------------- next part -------------- An HTML attachment was scrubbed... URL: From flub at devork.be Fri Nov 14 00:34:11 2014 From: flub at devork.be (Floris Bruynooghe) Date: Thu, 13 Nov 2014 23:34:11 +0000 Subject: [pytest-dev] Enabling viewing of rewritten AST trees as Python In-Reply-To: References: Message-ID: Hi Tom, On 9 November 2014 19:18, Tom Viner wrote: > The four options I can think of to gain access to this variable: > - monkey patch rewrite.rewrite_asserts > - edit rewrite_asserts and paste in the logging code above: > -- temporarily, by each developer, on each occasion required > -- permanently, but only running when a certain debug flag is active > - expose the rewritten AST via a new hook > > monkey patch is my preferred option, because if I can make it work, a plugin > could be written. I just need some help to find an appropriate hook to do > the monkey patching before pytest gets going with the AssertionRewritingHook > and calls rewrite_asserts. Without knowledge of an early-running hook, I > could only get this working by patching rewrite_asserts then calling > pytest.main myself, see > https://gist.github.com/tomviner/13c95cdb1e159028fc0b Would this work by using monkeypatching, but using a method on _pytest.pytester.TmpTestDir which wraps around inline_run or inline_runsource? That would seem like a fairly reasonable approach. It could simply skip with an appropriate message if the meta lib is importable and print the reversed python code to stdout for capsys to collect. If you would like to go the full plugin way I believe you just need to hook in before collection happens. I imagine pytest_configure would be a reasonable hook for that, but have to admit I haven't tried it. Regards, Floris From pupssman at gmail.com Fri Nov 14 23:53:10 2014 From: pupssman at gmail.com (Ivan Kalinin) Date: Sat, 15 Nov 2014 02:53:10 +0400 Subject: [pytest-dev] Extending xdist Message-ID: Hello there, fellow developers. I am now trying to add xdist intergration capabilities for my plugin, pytest-allure-adaptor and I've run into substantial problems with understanding xdist. Efforts so far are discussed here: https://github.com/allure-framework/allure-python/issues/8 . As far as the reverse-enginering leeds me, xdist is running a `master` process and `slave` processes, transmitting pytest's Items to slaves and receiving test results from those. It does so by hand-picking into the pytest hooks, calling some of them on the remote site and some on the master side, effectively achieving the `separated` pytest session. However, pytest-allure-adaptor, being a report-generation-kind plugin, relies significantly on the hook specification and call order, and xdist's hook manipulation renders the allure adaptor incapable of working properly. Here come the questions: 1) Is there any kind of documentation / guide lines / examples of xdist's internal architecture and integration? 2) Are there any planned activity on xdist internals? Or any kind of overhaul? Do xdist developers consider its use of hook an API? Is it sane to rely on xdist internal workings retaining campatibility for at least some time? 3) What's recommended / renown way of developing an xdist-capable pytest plugin? 4) Is it sane to rely on pytest hook call order at all? What is the most other-plugin-friendly way to override / modify pytest hooks? Hope for best and thanks in advance for your time. Best, Ivan. -------------- next part -------------- An HTML attachment was scrubbed... URL: From bubenkoff at gmail.com Sat Nov 15 00:34:10 2014 From: bubenkoff at gmail.com (Anatoly Bubenkov) Date: Sat, 15 Nov 2014 00:34:10 +0100 Subject: [pytest-dev] Extending xdist In-Reply-To: References: Message-ID: On 14 November 2014 23:53, Ivan Kalinin wrote: > Hello there, fellow developers. > > I am now trying to add xdist intergration capabilities for my plugin, > pytest-allure-adaptor and I've run into substantial problems with > understanding xdist. Efforts so far are discussed here: > https://github.com/allure-framework/allure-python/issues/8 . > > can you please translate those to a concrete set of problems in english? it's really hard for me to understand what is the problem, even though im a native russian speaker :) > Here come the questions: > > 1) Is there any kind of documentation / guide lines / examples of xdist's > internal architecture and integration? > it depends on the level you want to integrate - simple things you can look up in pytest-bdd plugin, we made it so that reporting works both with and without pytest-xdist > > -- Anatoly Bubenkov -------------- next part -------------- An HTML attachment was scrubbed... URL: From flub at devork.be Sat Nov 15 00:52:03 2014 From: flub at devork.be (Floris Bruynooghe) Date: Fri, 14 Nov 2014 23:52:03 +0000 Subject: [pytest-dev] yield_fixture still experimental? In-Reply-To: References: Message-ID: On 4 Nov 2014 13:38, "Bruno Oliveira" wrote: > > Is yield_fixture still experimental? Are there any plans to change it for 2.7, or later? Personally I would like to see the normal fixture become the yield fixture. I've not yet been convinced that this would be a problem, but given that both Holger and Ronny disagreed last time I argued for this on IRC maybe I'm missing something obvious? > I ask because a colleague of mind was somewhat cautious about using them because of their "experimental" status. I imagine the current API for them will not go away anymore as it's seen a lot of uptake. Only a long deprecation cycle would work now. So the docs can probably be updated. Floris -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicoddemus at gmail.com Sun Nov 16 06:46:47 2014 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Sun, 16 Nov 2014 03:46:47 -0200 Subject: [pytest-dev] yield_fixture still experimental? In-Reply-To: References: Message-ID: On Fri, Nov 14, 2014 at 9:52 PM, Floris Bruynooghe wrote: > I imagine the current API for them will not go away anymore as it's seen a > lot of uptake. Only a long deprecation cycle would work now. So the docs > can probably be updated. > Thanks Floris! That's what I was thinking. Cheers, -------------- next part -------------- An HTML attachment was scrubbed... URL: From mahmoud at paypal.com Sat Nov 22 01:24:25 2014 From: mahmoud at paypal.com (Hashemi, Mahmoud) Date: Sat, 22 Nov 2014 00:24:25 +0000 Subject: [pytest-dev] Exemplary py.test usage in open-source? Message-ID: <539667A1C8758B488DEE4541DC352B4252BD6AAC@PHX-EXRDA-S61.corp.ebay.com> Hey pytest-dev, Had a casual question for you all. We all know py.test is a great testing framework with a lot of top-notch documentation and plugins; several teams here at eBay/PayPal make good use of it (some of which is viewable on eBay's public Github). But some folks around here learn best from example. So in the spirit of open-source, would anyone mind sharing some Github/Bitbucket links to codebases that leverage py.test to its fullest? I'm especially interested in larger, more "enterprise" codebases with multifaceted tests. Werkzeug, devpi, and other infrastructural code is great, please share your faves along those lines, but I'm thinking more in the vein of Mozilla's tests (https://github.com/mozilla?query=tests). Thanks! Mahmoud Lead Developer, Python Infrastructure at eBay/PayPal https://github.com/mahmoud -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Mon Nov 24 09:04:00 2014 From: holger at merlinux.eu (holger krekel) Date: Mon, 24 Nov 2014 08:04:00 +0000 Subject: [pytest-dev] Exemplary py.test usage in open-source? In-Reply-To: <539667A1C8758B488DEE4541DC352B4252BD6AAC@PHX-EXRDA-S61.corp.ebay.com> References: <539667A1C8758B488DEE4541DC352B4252BD6AAC@PHX-EXRDA-S61.corp.ebay.com> Message-ID: <20141124080400.GM25600@merlinux.eu> Hey Hashemi, On Sat, Nov 22, 2014 at 00:24 +0000, Hashemi, Mahmoud wrote: > Hey pytest-dev, > > Had a casual question for you all. We all know py.test is a great testing framework with a lot of top-notch documentation and plugins; several teams here at eBay/PayPal make good use of it (some of which is viewable on eBay's public Github). > > But some folks around here learn best from example. So in the spirit of open-source, would anyone mind sharing some Github/Bitbucket links to codebases that leverage py.test to its fullest? I'm especially interested in larger, more "enterprise" codebases with multifaceted tests. Werkzeug, devpi, and other infrastructural code is great, please share your faves along those lines, but I'm thinking more in the vein of Mozilla's tests (https://github.com/mozilla?query=tests). > > Thanks! I'd also be interested in hearing back! http://pytest.org/latest/projects.html hasn't been updated for a while either FWIW. If there is some kind of github/bitbucket/code search engine maybe that could be used to automatically discover pytest using projects and sort them according to size or so? If somebody is willing to try that, that'd simplify communication i guess :) holger From jh at solute.de Mon Nov 24 13:57:22 2014 From: jh at solute.de (Jens Hoffmann) Date: Mon, 24 Nov 2014 13:57:22 +0100 Subject: [pytest-dev] pytest_plugin with session scope fixture Message-ID: <54732B32.1000307@solute.de> Hi everybody, I have a problem concering the following setup: - I have written a pytest plugin defining a session scope fixture - in my test files - say ./test/a/test_foo.py and ./test/b/test_bar.py I ``import'' the pytest plugin via declaring the pytest_plugins variable: pytest_plugins = 'my.cool.plugin' So, there's no conftest.py - say below ./test, that declares my.cool.plugin ``globally'' for test/a and test/b What is the session scope in this setup when I run ``py.test test'' on the command line? Is the scope split in test/a and test/b or do we still have a ``real'' session scope for everything below test? My real but similar setup seems to split the scope into chunks of modules which contain test files which declare pytest_plugins... but may be I am wrong... Is that possible? Thank you for your answer, Jens From edisongustavo at gmail.com Mon Nov 24 16:51:18 2014 From: edisongustavo at gmail.com (Edison Gustavo Muenz) Date: Mon, 24 Nov 2014 13:51:18 -0200 Subject: [pytest-dev] Pytest fails with the wrong error when unicode_literals is present Message-ID: TL;DR; The eval() method of the Frame object is not ?inheriting? the __future__ imports from the frame. Long explanation Suppose I have a fixture in which I write my assert statements. If this fixture has strings on the assert statement then when the assert fails I get the wrong kind of errors. The problem is that the pytest assertion rewrite mechanism is calling eval() on the code that contains the frame To better illustrate what happens, see this gist The tests have the following output: - test_plugin_fixture - *fail* - test_conftest_fixture - *fail* - test_inline_fixture - pass - test_no_fixture - pass The output of *test_plugin_fixture* (omitted some output for brevity): $ pytest -k test_plugin_fixture msg = e.value.msg > assert msg.startswith('assert library. The ?wrong? behaviour can be seen on the run_pyframe_eval.py . I?ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - 64bits. ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From holger at merlinux.eu Mon Nov 24 17:18:49 2014 From: holger at merlinux.eu (holger krekel) Date: Mon, 24 Nov 2014 16:18:49 +0000 Subject: [pytest-dev] pytest_plugin with session scope fixture In-Reply-To: <54732B32.1000307@solute.de> References: <54732B32.1000307@solute.de> Message-ID: <20141124161849.GP25600@merlinux.eu> Hi Jens, On Mon, Nov 24, 2014 at 13:57 +0100, Jens Hoffmann wrote: > Hi everybody, > > I have a problem concering the following setup: > > - I have written a pytest plugin defining a session scope fixture > - in my test files - say ./test/a/test_foo.py and > ./test/b/test_bar.py I ``import'' the pytest plugin via declaring > the pytest_plugins variable: pytest_plugins = 'my.cool.plugin' > > > So, there's no conftest.py - say below ./test, that declares > my.cool.plugin ``globally'' for test/a and test/b > > > > What is the session scope in this setup when I run ``py.test test'' > on the command line? Is the scope split in test/a and test/b or do > we still have a ``real'' session scope for everything below test? "my.cool.plugin" will have a "global" session scope which means test/a and test/b will share the same session-scoped fixture instances. > My real but similar setup seems to split the scope into chunks of > modules which contain test files which declare pytest_plugins... but > may be I am wrong... > Is that possible? I don't understand the question, sorry. Also I am not sure why you use the indirection of "pytest_plugins". why do conftest.py files or setuptools-installed plugijns not work for you? holger From mahmoud at paypal.com Tue Nov 25 01:05:34 2014 From: mahmoud at paypal.com (Hashemi, Mahmoud) Date: Tue, 25 Nov 2014 00:05:34 +0000 Subject: [pytest-dev] Exemplary py.test usage in open-source? In-Reply-To: <20141124080400.GM25600@merlinux.eu> References: <539667A1C8758B488DEE4541DC352B4252BD6AAC@PHX-EXRDA-S61.corp.ebay.com> <20141124080400.GM25600@merlinux.eu> Message-ID: <539667A1C8758B488DEE4541DC352B4252BD6F00@PHX-EXRDA-S61.corp.ebay.com> Hehe, sorry for the top post, work computer. I did actually try searching conftest.py on Github, but there were altogether too many results :) Didn't seem like there was a solid sorting mechanism either. The Mozilla tests are particularly interesting because they use Python to test non-Python codebases, which is a major area of interest in the increasingly diverse PayPal engineering community. Those sorts of py.test conventions really resonate for the corporate/product types :) Also, small note, I'd probably update the Projects page link to the Mozilla Github so the code is more readily showcased. Otherwise, the Projects page is fantastic :) Hope to hear more soon! Mahmoud -----Original Message----- From: holger krekel [mailto:holger at merlinux.eu] Sent: Monday, November 24, 2014 12:04 AM To: Hashemi, Mahmoud Cc: pytest-dev at python.org; mahmoud at hatnote.com Subject: Re: [pytest-dev] Exemplary py.test usage in open-source? Hey Hashemi, On Sat, Nov 22, 2014 at 00:24 +0000, Hashemi, Mahmoud wrote: > Hey pytest-dev, > > Had a casual question for you all. We all know py.test is a great testing framework with a lot of top-notch documentation and plugins; several teams here at eBay/PayPal make good use of it (some of which is viewable on eBay's public Github). > > But some folks around here learn best from example. So in the spirit of open-source, would anyone mind sharing some Github/Bitbucket links to codebases that leverage py.test to its fullest? I'm especially interested in larger, more "enterprise" codebases with multifaceted tests. Werkzeug, devpi, and other infrastructural code is great, please share your faves along those lines, but I'm thinking more in the vein of Mozilla's tests (https://github.com/mozilla?query=tests). > > Thanks! I'd also be interested in hearing back! http://pytest.org/latest/projects.html hasn't been updated for a while either FWIW. If there is some kind of github/bitbucket/code search engine maybe that could be used to automatically discover pytest using projects and sort them according to size or so? If somebody is willing to try that, that'd simplify communication i guess :) holger From jh at solute.de Tue Nov 25 07:23:38 2014 From: jh at solute.de (Jens Hoffmann) Date: Tue, 25 Nov 2014 07:23:38 +0100 Subject: [pytest-dev] pytest_plugin with session scope fixture In-Reply-To: <20141124161849.GP25600@merlinux.eu> References: <54732B32.1000307@solute.de> <20141124161849.GP25600@merlinux.eu> Message-ID: <5474206A.2060208@solute.de> Hi Holger, conftest.py/setuptools works just fine, but many tests require the absence of the plugin because of importing/dependency problems. So we decided to explicitly declare to use the plugin in every test file (also I find this to be a little less indirection). But anyway, things are as you say; even the so included plugin declares its session scope fixtures globally for every test; I doubted on this for just a moment, because my test-suite does strange things right now (apparently since I introduced that per-file pytest-plugin-declaration). But thank you anyway for your quick response! Jens On 11/24/2014 05:18 PM, holger krekel wrote: > Hi Jens, > > On Mon, Nov 24, 2014 at 13:57 +0100, Jens Hoffmann wrote: >> Hi everybody, >> >> I have a problem concering the following setup: >> >> - I have written a pytest plugin defining a session scope fixture >> - in my test files - say ./test/a/test_foo.py and >> ./test/b/test_bar.py I ``import'' the pytest plugin via declaring >> the pytest_plugins variable: pytest_plugins = 'my.cool.plugin' >> >> >> So, there's no conftest.py - say below ./test, that declares >> my.cool.plugin ``globally'' for test/a and test/b >> >> >> >> What is the session scope in this setup when I run ``py.test test'' >> on the command line? Is the scope split in test/a and test/b or do >> we still have a ``real'' session scope for everything below test? > "my.cool.plugin" will have a "global" session scope which means test/a > and test/b will share the same session-scoped fixture instances. > >> My real but similar setup seems to split the scope into chunks of >> modules which contain test files which declare pytest_plugins... but >> may be I am wrong... >> Is that possible? > I don't understand the question, sorry. > Also I am not sure why you use the indirection of "pytest_plugins". > why do conftest.py files or setuptools-installed plugijns not work for you? > > holger From holger at merlinux.eu Tue Nov 25 10:03:40 2014 From: holger at merlinux.eu (holger krekel) Date: Tue, 25 Nov 2014 09:03:40 +0000 Subject: [pytest-dev] Exemplary py.test usage in open-source? In-Reply-To: <539667A1C8758B488DEE4541DC352B4252BD6F00@PHX-EXRDA-S61.corp.ebay.com> References: <539667A1C8758B488DEE4541DC352B4252BD6AAC@PHX-EXRDA-S61.corp.ebay.com> <20141124080400.GM25600@merlinux.eu> <539667A1C8758B488DEE4541DC352B4252BD6F00@PHX-EXRDA-S61.corp.ebay.com> Message-ID: <20141125090340.GR25600@merlinux.eu> On Tue, Nov 25, 2014 at 00:05 +0000, Hashemi, Mahmoud wrote: > Hehe, sorry for the top post, work computer. I did actually try searching conftest.py on Github, but there were altogether too many results :) Didn't seem like there was a solid sorting mechanism either. > > The Mozilla tests are particularly interesting because they use Python to test non-Python codebases, which is a major area of interest in the increasingly diverse PayPal engineering community. Those sorts of py.test conventions really resonate for the corporate/product types :) Also, small note, I'd probably update the Projects page link to the Mozilla Github so the code is more readily showcased. Otherwise, the Projects page is fantastic :) I happily accept PRs listing projects. Please add your own project contexts as that would be interesting for others as well i guess :) As far as testing non-python codebases goes, these two integrate C/C++: https://pypi.python.org/pypi/pytest-cpp http://pytest-c-testrunner.readthedocs.org/en/latest/ Many are using pytest to test command line scripts (which are not neccessarily written in Python) and pytest itself has many tests written against it this way. Didn't get around to release a "pytest-cli" plugin i have started somewhere. > Hope to hear more soon! same here :) holger From nicoddemus at gmail.com Tue Nov 25 13:49:44 2014 From: nicoddemus at gmail.com (Bruno Oliveira) Date: Tue, 25 Nov 2014 10:49:44 -0200 Subject: [pytest-dev] Pytest fails with the wrong error when unicode_literals is present In-Reply-To: References: Message-ID: Hi everyone, Besides the problem that reinterpret is executing code in a context without unicode_literals enabled, this also showcases that when moving a fixture with has asserts on it from a test file to a conftest file, asserts will not be rewritten anymore and reinterpretation will be used instead, which might cause unexpected issues, as illustrated by Gustavo. The first issue looks like a bug to me, not sure what to do about this second point. Any thoughts? Cheers, On Mon, Nov 24, 2014 at 1:51 PM, Edison Gustavo Muenz < edisongustavo at gmail.com> wrote: > TL;DR; > > The eval() method of the Frame object is not ?inheriting? the __future__ > imports from the frame. > Long explanation > > Suppose I have a fixture in which I write my assert statements. If this > fixture has strings on the assert statement then when the assert fails I > get the wrong kind of errors. > > The problem is that the pytest assertion rewrite mechanism is calling > eval() on the code that contains the frame > > To better illustrate what happens, see this gist > > > The tests > > have the following output: > > - test_plugin_fixture - *fail* > - test_conftest_fixture - *fail* > - test_inline_fixture - pass > - test_no_fixture - pass > > The output of *test_plugin_fixture* (omitted some output for brevity): > > $ pytest -k test_plugin_fixture > > msg = e.value.msg > > assert msg.startswith('assert E AssertionError: ValueError: ValueAndString(2, 'lkajflaskjfalskf') << Error! > > While debugging I found out that the ValueAndString constructor was being > called 3 times, instead of 2. The 3rd call happened in code.py:100, which > is part of the py lib library. The > ?wrong? behaviour can be seen on the run_pyframe_eval.py > > . > > I?ve tested this code on Pytest 2.6.4 with Python 2.7.8 in Windows 7 - > 64bits. > ? > > _______________________________________________ > pytest-dev mailing list > pytest-dev at python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nikolaus at rath.org Wed Nov 26 21:21:04 2014 From: Nikolaus at rath.org (Nikolaus Rath) Date: Wed, 26 Nov 2014 12:21:04 -0800 Subject: [pytest-dev] Help with parametrized fixtures Message-ID: <87oartg1cf.fsf@rath.org> Hello, I have a parametrized fixture foo, and some test cases that use it: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... return foo_f def test_1(foo): # ... def test_2(foo): # .... Now, I'd like to add an additional test, but running it only makes sense for some of the fixture parameters. So far I've solved this by providing the parameter value as a fixture attribute and skipping "unsupported" values: @pytest.fixture(params=(1,2,3,4,5)) def foo(request): # ... foo_f.param = request.param return foo_f def test_3(foo): if foo.param not in (1,4,5): raise pytest.skip("doesn't make sense") This works, but I don't like it very much because it means that the test suite can never be executed without skipping some tests. I'd rather reserve skipping for cases where the test could in principle be executed, but for some reason cannot work at the moment (e.g. because there's no internet, or a required utility is not installed). Is there a way to solve the above scenario without skipping tests? Ideally, I'd be able to do something like this: @only_for_param((1,4,5)) def test_3(foo): ... Thanks! -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F ?Time flies like an arrow, fruit flies like a Banana.? From carl at oddbird.net Wed Nov 26 22:31:03 2014 From: carl at oddbird.net (Carl Meyer) Date: Wed, 26 Nov 2014 14:31:03 -0700 Subject: [pytest-dev] Help with parametrized fixtures In-Reply-To: <87oartg1cf.fsf@rath.org> References: <87oartg1cf.fsf@rath.org> Message-ID: <54764697.2090802@oddbird.net> Hi Nikolaus, On 11/26/2014 01:21 PM, Nikolaus Rath wrote: > I have a parametrized fixture foo, and some test cases that use it: > > @pytest.fixture(params=(1,2,3,4,5)) > def foo(request): > # ... > return foo_f > > def test_1(foo): > # ... > > def test_2(foo): > # .... > > Now, I'd like to add an additional test, but running it only makes sense > for some of the fixture parameters. So far I've solved this by providing > the parameter value as a fixture attribute and skipping "unsupported" values: > > @pytest.fixture(params=(1,2,3,4,5)) > def foo(request): > # ... > foo_f.param = request.param > return foo_f > > def test_3(foo): > if foo.param not in (1,4,5): > raise pytest.skip("doesn't make sense") > > > This works, but I don't like it very much because it means that the test > suite can never be executed without skipping some tests. I'd rather > reserve skipping for cases where the test could in principle be > executed, but for some reason cannot work at the moment (e.g. because > there's no internet, or a required utility is not installed). > > Is there a way to solve the above scenario without skipping tests? > > Ideally, I'd be able to do something like this: > > @only_for_param((1,4,5)) > def test_3(foo): > ... I think in order to do this you'll need to use the `pytest_generate_tests` hook: http://pytest.org/latest/parametrize.html#pytest-generate-tests I might be wrong (haven't had to use pytest_generate_tests much), but I don't think you'll be able to keep the simple parametrized fixture and layer this on top; I don't think pytest_generate_tests can tweak the fixture params per-test. You'll need to remove the fixture parametrization and do it instead via pytest_generate_tests, using indirect=True. Carl -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Nikolaus at rath.org Thu Nov 27 23:22:55 2014 From: Nikolaus at rath.org (Nikolaus Rath) Date: Thu, 27 Nov 2014 14:22:55 -0800 Subject: [pytest-dev] Help with parametrized fixtures In-Reply-To: <54764697.2090802@oddbird.net> (Carl Meyer's message of "Wed, 26 Nov 2014 14:31:03 -0700") References: <87oartg1cf.fsf@rath.org> <54764697.2090802@oddbird.net> Message-ID: <87a93c8erk.fsf@vostro.rath.org> Carl Meyer writes: > Hi Nikolaus, > > On 11/26/2014 01:21 PM, Nikolaus Rath wrote: >> I have a parametrized fixture foo, and some test cases that use it: >> >> @pytest.fixture(params=(1,2,3,4,5)) >> def foo(request): >> # ... >> return foo_f >> >> def test_1(foo): >> # ... >> >> def test_2(foo): >> # .... >> >> Now, I'd like to add an additional test, but running it only makes sense >> for some of the fixture parameters. So far I've solved this by providing >> the parameter value as a fixture attribute and skipping "unsupported" values: >> >> @pytest.fixture(params=(1,2,3,4,5)) >> def foo(request): >> # ... >> foo_f.param = request.param >> return foo_f >> >> def test_3(foo): >> if foo.param not in (1,4,5): >> raise pytest.skip("doesn't make sense") >> >> This works, but I don't like it very much because it means that the test >> suite can never be executed without skipping some tests. I'd rather >> reserve skipping for cases where the test could in principle be >> executed, but for some reason cannot work at the moment (e.g. because >> there's no internet, or a required utility is not installed). >> >> Is there a way to solve the above scenario without skipping tests? >> >> Ideally, I'd be able to do something like this: >> >> @only_for_param((1,4,5)) >> def test_3(foo): >> ... > > I think in order to do this you'll need to use the > `pytest_generate_tests` hook: > http://pytest.org/latest/parametrize.html#pytest-generate-tests Thanks for the link! The following seems to work nicely: #!/usr/bin/env python3 import pytest @pytest.fixture() def bar(request): s = 'bar-%d' % request.param print('preparing', s) return s def test_one(bar): pass def test_two(bar): pass test_two.test_with = (2,3) def pytest_generate_tests(metafunc): params = (1,2,3,4) if not 'bar' in metafunc.fixturenames: return test_with = getattr(metafunc.function, 'test_with', None) if test_with: params = test_with metafunc.parametrize("bar", params, indirect=True) Unfortunately, it seems there are several issues with parametrizing fixtures these way that are probably going to bite me as things get more complex: https://bitbucket.org/hpk42/pytest/issue/635 https://bitbucket.org/hpk42/pytest/issue/634 https://bitbucket.org/hpk42/pytest/issue/531 https://bitbucket.org/hpk42/pytest/issue/519 (Maybe mentioning them here gives them more attention :-). Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F ?Time flies like an arrow, fruit flies like a Banana.? From Nikolaus at rath.org Fri Nov 28 00:39:17 2014 From: Nikolaus at rath.org (Nikolaus Rath) Date: Thu, 27 Nov 2014 15:39:17 -0800 Subject: [pytest-dev] Access markers from pytest_generate_tests? Message-ID: <87wq6ggqmy.fsf@vostro.rath.org> Hello, Is there a way to access test markers sin the pytest_generate_tests hook? For example: @pytest.mark.bla def test_one(): # ... def pytest_generate_tests(metafunc): if metafunc_has_bla_mark(metafunc): # do something But what do you have to use for metafunc_has_bla_mark? Unfortunately metafunc.function does not have a `keywords` attribute... Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F ?Time flies like an arrow, fruit flies like a Banana.?