Mailman 3 September 2016 - pytest-dev

process for replacing yield tests with modern style
by Chris Dent Nov. 28, 2016

Nov. 28, 2016

Several years ago I developed a relative simple YAML-based way of doing HTTP tests. The processed turned yaml into tests and yielded them for pytest to collect. At the time yield was the pytest approved way of doing such things. That's here: https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py A few years later I took that idea and turned it into gabbi[1]. Because of local constraints (OpenStack) gabbi needed to be unittest, subunit and testrepository compliant and oriented. Only later, and in rather a hurry, did I add pytest support, using the yield style that I already knew. It's a bit of hacky mess because the tests are ordered unittest.TestCases inside a unittest.TestSuite that has fixtures in the scope of the TestSuite. Each yaml file becomes a TestSuite. For the pytest side of things, plugin code[2] uses the modifyitems hooks to get the fixtures working effectively. Some "driver"[3] code yields tests to a user-code[4] for loading the tests. I recognize that this is all a bit messy, but it does the job. Now, with the advent of pytest 3.x yield tests are deprecated and I'm not sure of the correct way to get the same behavior in modern-style. I'm aware of solutions for using yaml files themselves as the test files[5] but that would be a problem for existing deployed tests that use the style shown in [4] and would also require conftest.py or command line arguments and what I'm hoping to maintain is a way for unadorned discovery to work and for the gabbi tests to be fairly integrated with other tests. I can certainly migrate to the [5] style but I see this as a good opportunity to learn more of the details of how pytest generates tests and use it. Can anyone give me some pointers please? Thanks very much. [1] https://gabbi.readthedocs.io/ [2] https://github.com/cdent/gabbi/blob/master/gabbi/pytester.py [3] https://github.com/cdent/gabbi/blob/2bd1c803b71bd26be7822be87e249a811467b86… [4] https://github.com/cdent/gabbi/blob/master/gabbi/tests/test_gabbits_pytest.… [5] http://docs.pytest.org/en/latest/example/nonpython.html#a-basic-example-for… -- Chris Dent ┬─┬ノ( º _ ºノ) https://anticdent.org/ freenode: cdent tw: @anticdent

4 7

Moving pytest-rerunfailures to pytest-dev
by Florian Pilz Oct. 27, 2016

Oct. 27, 2016

Hi, I’m asking whether `pytest-rerunfailures` should be moved to the `pytest-dev` organisation (https://github.com/gocept/pytest-rerunfailures <https://github.com/gocept/pytest-rerunfailures>). Since we (gocept) only rarely take time to work on issues / pull requests of `pytest-rerunfailures` we would prefer to move the extremely useful plugin to `pytest-dev`. There we still can work on it, but are not the only ones taking care of the project. So if you think `pytest-rerunfailures` is a great addition to `pytest` and should be moved to the pytest-dev organization, let us know. :) Best wishes Florian ––––––––– Florian Pilz · fp(a)gocept.com · Software Developer gocept gmbh & co. kg · Forsterstraße 29 · 06112 Halle (Saale) · Germany http://gocept.com <http://gocept.com/> · Tel +49 345 122 9889 7 Python, Pyramid, Plone, Zope · consulting, development, hosting, operations

3 8

Proposal: python namespacign for markings/marker objects
by Ronny Pfannschmidt Oct. 20, 2016

Oct. 20, 2016

Hi all, while trying to turn various internal markers of a work project into public plugins, i noticed a very plain problems - different other plugins used the same generic marker name for different purposes/intents such a flat namespace really doesn't scale as such i would like to propose having marker types and objects as importable objects for example import pytest from pytest_bugzilla import Blocker @pytest.mark(Blocker(123)) def test_fun(): pass that way we can do both, reuse python name-spacing *and* use more meaningful objects for markings -- Ronny -- Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander

5 9

How to repeat tests with a setup-action before each repitition
by Emil Petersen Oct. 4, 2016

Oct. 4, 2016

Hello pytest-dev, I am looking to repeat some of the tests in my test suite, and want to call a certain action before each repetition. To be clear, I want to: 1) First, run all the tests marked with repeat, as-is 2) Then, take a specific action, in this case rebooting a device 3) Then, repeat each test marked with repeat 4) Then, repeat steps 2-3 the amount of times specified. The reason I want to run step #2 before every “repeat” is because step #2 is expensive, but needs to be performed before the tests are called. So my ideal code might look something like this (not real code): @pytest.mark.repeat def test_something() check_something == last_result_from_file() @pytest.mark.repeat def test_something_else: something_else() def expensive_setup(): set_up() Using the above as an example, I want to run test_something and test_something_else, then expensive_setup, then the two tests again, then expensive_setup, etc. So only one call to expensive_setup for each run including test_something() and test_something_else() What would be a sensible way to go about this? As an addendum, it is ideal if each test is only reported as -one- test, rather than one test being generated for each run. But this last thing I imagine/hope I can do in some of the reporting hooks. Ideally, if any one call to the functions ends up generating a failure or error, it would be able to fail, and abort any attempts at subsequently running (I.e. failures should be final). However, I’ve found no satisfactory way to do the above. Can anyone provide some pointers about a good way to go about this? In case you’re curious about why the tests need to be repeated, it’s to hunt for race conditions, and other failures which may occur sometimes. Thank you for your time. Kind Regards, -- Emil Petersen | Software Test Engineer <https://www.thalmic.com/> T: 1-888-777-2546 x692 <> E: emil.petersen(a)thalmic.com <mailto:victoria.baskerville@thalmic.com> -- ---- CONFIDENTIALITY NOTICE: This communication, and any attachments thereto, is confidential and intended solely for use by the recipient(s) named above.

2 2

pytest 3.0.3 released!
by Bruno Oliveira Sept. 29, 2016

Sept. 29, 2016

Hello everyone, pytest 3.0.3 has just been released to PyPI. This release fixes some regressions and bugs reported in the last version, being a drop-in replacement. To upgrade:: pip install --upgrade pytest The changelog is available at http://doc.pytest.org/en/latest/changelog.html . Thanks to all who contributed to this release, among them: * Bruno Oliveira * Florian Bruhin * Floris Bruynooghe * Huayi Zhang * Lev Maximov * Raquel Alegre * Ronny Pfannschmidt * Roy Williams * Tyler Goodlet * mbyt Happy testing, The pytest Development Team

1 0

the sorry state of pylib tests and pylib
by Ronny Pfannschmidt Sept. 27, 2016

Sept. 27, 2016

Hi everyone, while fixing up some pylib tests i noticed a set of massive issues a) all py.code related tests are completely broken on pytest 3.0 b) the svn related tests are completely broken and outdated (can we kill it?) c) depending on environment there are massive issues with pylib path encodings as far as i can tell thats just scratching the surface im not yet sure what to do about it, but some thing has to be done ^^ -- Ronny

4 12

pytest-warnings in 3.1
by Bruno Oliveira Sept. 22, 2016

Sept. 22, 2016

Hi everyone, During the sprint Florian Schulze developed pytest-warnings[1], and later we decided to merge that into pytest 3.1. I was trying to remember: did anybody volunteer to do that? Cheers, [1] https://github.com/fschulze/pytest-warnings

3 5

Re: [pytest-dev] tox is now on github
by Tom Viner Sept. 20, 2016

Sept. 20, 2016

Hi Ronny, Any plans to move detox as well? I've found that along with a local devpi as a cache it can really speed up tox runs that have many envs. It currently lives at: https://bitbucket.org/hpk42/detox Either way, I note that the PyPi description ( https://pypi.python.org/pypi/detox/ ) is now outdated as it links to the tox issues on bitbucket, so I've made a PR to fix that: https://bitbucket.org/hpk42/detox/pull-requests/5/ Cheers, Tom Date: Sat, 17 Sep 2016 22:14:09 +0200 > From: Ronny Pfannschmidt <opensource(a)ronnypfannschmidt.de> > Hello everyone, > > i am pleased to announce that after some setbacks > we now have finished the repository and > issue tracker migration for tox to github. > > its new home is https://github.com/tox-dev/tox > > due to the setbacks i ended up deferring various detail works (like > travis/readthedocs) > those will be finished during the next few days. > > even before the announcement the new place already started brimming with > activity > > > -- Ronny > >

1 0

tox is now on github
by Ronny Pfannschmidt Sept. 18, 2016

Sept. 18, 2016

Hello everyone, i am pleased to announce that after some setbacks we now have finished the repository and issue tracker migration for tox to github. its new home is https://github.com/tox-dev/tox due to the setbacks i ended up deferring various detail works (like travis/readthedocs) those will be finished during the next few days. even before the announcement the new place already started brimming with activity -- Ronny

2 3

the move of pylib to github is technically finished
by Ronny Pfannschmidt Sept. 17, 2016

Sept. 17, 2016

Hi all, i finished the move to github and did a few simple fixups in the process i learned various details, and fixed bugs in the migration tool now i feel confident i can do the same for tox in the afternoon. i'll finish up the detail works with pylib and create issues wrt its updating to pytest 3.x then i#ll focus on tox -- Ronny

2 1