Mailman 3 October 2016 - pytest-dev

t-shirts and stickers now out, thanks Florian!
by holger krekel Nov. 29, 2016

Nov. 29, 2016

Hi all, Florian and Regina visited last weekend, did most of the work on sending out shirts and stickers for both pytest and qutebrowser. Many thanks to Florian who spend almost a week alltogether preparing the shirts, stickers, figuring out and mostly doing all the logistics. I just brought the last packages to the post office and so everything should be on its way now. I'll leave it to him to tell about things to watch out for next time and summarize the experience. After the last costs are claimed i expect there to be around 4-5K EUR left of the original funding FYI. It's not my money so feel free to discuss how to use it. If nothing else it probably can be used for the next sprint or so. best, holger P.S.: Could someone help with setting up appveyor/windows-testing for tox's new https://github.com/tox-dev/tox repository? Would really appreciate it.

7 12

process for replacing yield tests with modern style
by Chris Dent Nov. 28, 2016

Nov. 28, 2016

Several years ago I developed a relative simple YAML-based way of doing HTTP tests. The processed turned yaml into tests and yielded them for pytest to collect. At the time yield was the pytest approved way of doing such things. That's here: https://github.com/tiddlyweb/tiddlyweb/blob/master/test/http_runner.py A few years later I took that idea and turned it into gabbi[1]. Because of local constraints (OpenStack) gabbi needed to be unittest, subunit and testrepository compliant and oriented. Only later, and in rather a hurry, did I add pytest support, using the yield style that I already knew. It's a bit of hacky mess because the tests are ordered unittest.TestCases inside a unittest.TestSuite that has fixtures in the scope of the TestSuite. Each yaml file becomes a TestSuite. For the pytest side of things, plugin code[2] uses the modifyitems hooks to get the fixtures working effectively. Some "driver"[3] code yields tests to a user-code[4] for loading the tests. I recognize that this is all a bit messy, but it does the job. Now, with the advent of pytest 3.x yield tests are deprecated and I'm not sure of the correct way to get the same behavior in modern-style. I'm aware of solutions for using yaml files themselves as the test files[5] but that would be a problem for existing deployed tests that use the style shown in [4] and would also require conftest.py or command line arguments and what I'm hoping to maintain is a way for unadorned discovery to work and for the gabbi tests to be fairly integrated with other tests. I can certainly migrate to the [5] style but I see this as a good opportunity to learn more of the details of how pytest generates tests and use it. Can anyone give me some pointers please? Thanks very much. [1] https://gabbi.readthedocs.io/ [2] https://github.com/cdent/gabbi/blob/master/gabbi/pytester.py [3] https://github.com/cdent/gabbi/blob/2bd1c803b71bd26be7822be87e249a811467b86… [4] https://github.com/cdent/gabbi/blob/master/gabbi/tests/test_gabbits_pytest.… [5] http://docs.pytest.org/en/latest/example/nonpython.html#a-basic-example-for… -- Chris Dent ┬─┬ノ( º _ ºノ) https://anticdent.org/ freenode: cdent tw: @anticdent

4 7

T-Shirt has arrived!
by Omar Kohl Nov. 3, 2016

Nov. 3, 2016

Hey, the 2016 Sprint T-Shirt arrived today! Thanks a lot, very cool :-) Omar

10 9

Moving pytest-rerunfailures to pytest-dev
by Florian Pilz Oct. 27, 2016

Oct. 27, 2016

Hi, I’m asking whether `pytest-rerunfailures` should be moved to the `pytest-dev` organisation (https://github.com/gocept/pytest-rerunfailures <https://github.com/gocept/pytest-rerunfailures>). Since we (gocept) only rarely take time to work on issues / pull requests of `pytest-rerunfailures` we would prefer to move the extremely useful plugin to `pytest-dev`. There we still can work on it, but are not the only ones taking care of the project. So if you think `pytest-rerunfailures` is a great addition to `pytest` and should be moved to the pytest-dev organization, let us know. :) Best wishes Florian ––––––––– Florian Pilz · fp(a)gocept.com · Software Developer gocept gmbh & co. kg · Forsterstraße 29 · 06112 Halle (Saale) · Germany http://gocept.com <http://gocept.com/> · Tel +49 345 122 9889 7 Python, Pyramid, Plone, Zope · consulting, development, hosting, operations

3 8

Re: [pytest-dev] [TIP] instantiate class and use in all tests
by James Oct. 26, 2016

Oct. 26, 2016

On 2016-10-26 11:01 AM, holger krekel wrote: > James, as you have posted several questions already wrt to pytest may i suggest you rather subscribe and start posting to: > > https://mail.python.org/mailman/listinfo/pytest-dev > > Moreover, your examples so far contained some glitches which indicate you are not that experienced with Python yet. That's totally fine but i suggest you try to keep things as simple as you can before using all kinds of Python features. > > As to the mail below you can instantiate a class from a fixture and provide that to a test: > > class DoStuff(): > def __init__(self, a,b): > self.a=a > self.b=b > > def log(self, msg): > # do logging stuff > pass > > @pytest.fixture > def do_stuff(a, b): > return DoStuff(a, b) > > best, > holger I still can't make it work: =================================== FAILURES =================================== ______________________________ TestAclass.test_1 _______________________________ self = <testscript.TestAclass instance at 0x7f87395bffc8> request = <FixtureRequest for <Function 'test_1'>> def test_1(self,request): > ds=do_stuff(a,b) E NameError: global name 'do_stuff' is not defined testscript.py:9: NameError Is it a python error? # conftest.py import pytest @pytest.fixture(scope='module') def need_a(request): value = request.config.getoption("A") if not value: pytest.skip('test needs -A option to run') return value @pytest.fixture(scope='module') def a(request): a = pytest.config.getoption("A") return a @pytest.fixture(scope='module') def b(request): b = pytest.config.getoption("B") return b def pytest_addoption(parser): parser.addoption("--A", action="store", default=None, help="a option") parser.addoption("--B", action="store", default=None, help="b option") class DoStuff(): def __init__(self,a,b): self.a=a self.b=b def log(self,msg): # do logging stuff pass @pytest.fixture(scope='module') def do_stuff(a,b): # I tried def do_stuff(a,b,request): return DoStuff(a, b) import pytest # testscript.py @pytest.mark.incremental class TestAclass(): def test_1(self,request): ds=do_stuff(a,b) print 'inside {0}-{1} step'.format(self.__class__.__name__,request.function.__name__) print 'a={0}, b={1}'.format(ds.a,ds.b) ds.log("we are here") pass

2 3

landing pytest.marked
by Ronny Pfannschmidt Oct. 24, 2016

Oct. 24, 2016

Hi everyone, i'd like to bring https://github.com/pytest-dev/pytest/pull/1921 back to attention it fixes a long-standing issue wrt putting mark on parameter values while properly separating the markers from the parameter value, it has intentionally limited scope of only dealing with marks, and provides a better solution to mark parameters, which also supports functions as parameters. the discussion on the pr itself has gone a bit too broad, and i'd like to call back to the solution of a concrete scoped problem perhaps marking the feature itself experimental -- Ronny -- Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander

1 0

Proposal: python namespacign for markings/marker objects
by Ronny Pfannschmidt Oct. 20, 2016

Oct. 20, 2016

Hi all, while trying to turn various internal markers of a work project into public plugins, i noticed a very plain problems - different other plugins used the same generic marker name for different purposes/intents such a flat namespace really doesn't scale as such i would like to propose having marker types and objects as importable objects for example import pytest from pytest_bugzilla import Blocker @pytest.mark(Blocker(123)) def test_fun(): pass that way we can do both, reuse python name-spacing *and* use more meaningful objects for markings -- Ronny -- Red Hat GmbH, http://www.de.redhat.com/, Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Michael Cunningham, Michael O'Neill, Eric Shander

5 9

Fwd: Re: Proposal: python namespacign for markings/marker objects
by Ronny Pfannschmidt Oct. 20, 2016

Oct. 20, 2016

forwarding as i send it with the wrong sender address -------- Forwarded Message -------- Subject: Re: [pytest-dev] Proposal: python namespacign for markings/marker objects Date: Wed, 19 Oct 2016 17:24:32 +0200 From: Ronny Pfannschmidt <ich(a)ronnypfannschmidt.de> To: Floris Bruynooghe <flub(a)devork.be>, pytest-dev <pytest-dev(a)python.org> On 19.10.2016 17:01, Floris Bruynooghe wrote: > On 7 September 2016 at 10:38, Ronny Pfannschmidt <rpfannsc(a)redhat.com> wrote: >> Hi all, >> >> while trying to turn various internal markers of a work project >> into public plugins, i noticed a very plain problems - different other >> plugins used the same generic marker name for different purposes/intents >> >> such a flat namespace really doesn't scale >> >> as such i would like to propose having marker types and objects as >> importable objects >> >> for example >> >> import pytest >> from pytest_bugzilla import Blocker >> >> @pytest.mark(Blocker(123)) >> def test_fun(): >> pass >> >> that way we can do both, reuse python name-spacing *and* use more meaningful >> objects for markings > I'm rather lukewarm for this proposal to be honest. We already have > an API which allows creating of mark objects for later re-use: > > blocker = pytest.mark.bugzilla_blocker > > @blocker > def test_fun(): > pass > > So instead of changing the way way pytest.mark behaves I think it > might be more interesting to look at alternative ways of creating > marker objects. E.g. if you could do: > > # pytest_bugzilla.py > class Blocker(pytest.MarkerObject): > def __init__(self, n): > pass > > # test_fun.py > from pytest_bugzilla import Blocker > > @Blocker(123) > def test_fun(): > pass > Then you can still import your marker and you also get more strict > namespacing since you created a unique object in your module. But the > benefit is that from the point of view of using markers not much has > changed. There's just a new way of creating markers. > > What do you reckon about such an approach? The main reason why i introduced the separation between objects and marking was to prevent a damn unreasonable mess. The objects that are the metadata put onto a function/object should be completely unaware of the py.test marking functionality, this is basics of separation of concern and prevents spaghetti behavior (like MarkInfo vs Markdecorator leaking into item.keywords and creating a fun painful mess) > > Finally, and I only just remembered, Holger has some lingering code > somewhere which does something like: > > @pytest.marker > def blocker(n): > return {'issue no': int(n)} # or whatever object you want your marker to be > > @pytest.mark.blocker(123) > def test_fun(): > pass fixtures and markers are not symmetric and i don't see a reason to make them be just for looking similar, currently they are *completely* different , and i think that bringing them closer to look more nice will make things MUCH worse. they are belonging to different categories of "things" with completely different desired behaviours. > This uses symmetry with how we create fixtures, which is nice. But it > does not solve the namespacing issue, which to some extend also exists > in fixtures. Just brainstorming to add something namespace like to > that would be: > > @pytest.marker(ns='bugzilla') > def blocker(n): > return int(n) > > @pytest.mark.bugzilla.blocker(123) > def test_fun(): > pass > > I admit this does not re-use the python module namespaces, which is > probably a downside. > i am strongly opposed to inventing a own name-spacing mechanism there, it would be another mess like the marker transfer, or the yield tests execute test code at collection time messes -- Ronny > Floris > _______________________________________________ > pytest-dev mailing list > pytest-dev(a)python.org > https://mail.python.org/mailman/listinfo/pytest-dev

3 3

Fwd: Re: ANN: pytest-external-blockers - prototype released
by Ronny Pfannschmidt Oct. 20, 2016

Oct. 20, 2016

forwarding as i send it with the wrong sender address -------- Forwarded Message -------- Subject: Re: [pytest-dev] ANN: pytest-external-blockers - prototype released Date: Wed, 19 Oct 2016 17:27:42 +0200 From: Ronny Pfannschmidt <ich(a)ronnypfannschmidt.de> To: Floris Bruynooghe <flub(a)devork.be>, Ronny Pfannschmidt <rpfannsc(a)redhat.com> CC: pytest-dev <pytest-dev(a)python.org> On 19.10.2016 17:21, Floris Bruynooghe wrote: > On 4 October 2016 at 18:18, Ronny Pfannschmidt <rpfannsc(a)redhat.com> wrote: >> Hi all, >> >> i'm happy to announce the release of pytest-external-blockers [0] >> it was created in order to be able to semantically track for what reasons a >> test was skipped >> >> a normal `pytest.skip` doesn't offer a semantic difference between "skipped >> for a local reason" and "skipped for a external reason" >> >> so we choose to pick a new verb. >> that way every dynamic "skip" that happens for a external reason (like >> database down, known to be broken tests, unavailable network service) can be >> identified. >> >> In future i want to extend this concept to xfail. >> >> I'm hoping for some harsh critique :) > This plugin is essentially trying to extend the possible outcomes, but > it needs to piggyback the skip exception for integration with the core > test running logic? It would be nice if you could subclass the > exception from a more generic pytest.OutcomeException or so instead. > Otherwise I think I don't dislike what the plugin itself does too much > ;-) that simply isn't/wasn't exposed at the time also the py.test internals are completely riddled with implicit expectations about outcomes, so much that it is scary i'd love to be able to spell things more granular in different manners as well -- Ronny > Floris > _______________________________________________ > pytest-dev mailing list > pytest-dev(a)python.org > https://mail.python.org/mailman/listinfo/pytest-dev

1 0

ANN: pytest-external-blockers - prototype released
by Ronny Pfannschmidt Oct. 19, 2016

Oct. 19, 2016

Hi all, i'm happy to announce the release of pytest-external-blockers [0] it was created in order to be able to semantically track for what reasons a test was skipped a normal `pytest.skip` doesn't offer a semantic difference between "skipped for a local reason" and "skipped for a external reason" so we choose to pick a new verb. that way every dynamic "skip" that happens for a external reason (like database down, known to be broken tests, unavailable network service) can be identified. In future i want to extend this concept to xfail. I'm hoping for some harsh critique :) [0] https://github.com/RonnyPfannschmidt/pytest-external-blockers https://pypi.python.org/pypi/pytest-external-blockers -- Ronny

2 1