Mailman 3 May 2013 - pytest-dev

Re: [pytest-dev] [pytest-rerunfailures] pytest-rerunfailures not using fixtures on reruns (#10)
by Leah Klearman July 16, 2013

July 16, 2013

Hi Holger and other py.test mavens, Bob has reported a problem with my py.test plugin pytest-rerunfailures [1] not re-running the setup before rerunning the test. Looking at my code, it has pytest_runtest_protocol() [2] looping on _pytest.runner.runtestprotocol() [3], which in turn runs the setup, the test, and the teardown. [1] https://github.com/klrmn/pytest-rerunfailures [2] https://github.com/klrmn/pytest-rerunfailures/blob/master/rerunfailures/plu… [3] https://bitbucket.org/hpk42/pytest/src/fdc28ac2029f1c0be1dac4991c2f1b014c39… I haven't taken the hours needed to get my head fully into py.test plugin development mode, but I'm not sure I can implement a fix at my layer. I'm hoping someone here will have some insight. Thanks, -Leah On Sun, Apr 14, 2013 at 6:29 PM, Bob Silverberg <notifications(a)github.com>wrote: > I just verified this behaviour myself with a simple test [1]. I see it > with both funcargs and fixtures, but I'm not sure if it has to do with the > plugin, or the way py.test works. It does inject the value into the test > method, but it doesn't rerun the fixture, so it seems like it is caching > the first run of the fixture and using that on subsequent runs. > > I'm not sure if this is something that the plugin can have any effect on, > or if it's just the way fixtures work. It is specified for this fixture > that it is scope='function', and perhaps py.test makes that happen by > checking the function name, which is, of course, the same for each run. I > did try removing the scope argument from the fixture but that had no > effect. > > Do you have any thoughts about this, @klrmn <https://github.com/klrmn>? > > [1] https://gist.github.com/bobsilverberg/5385035 > > — > Reply to this email directly or view it on GitHub<https://github.com/klrmn/pytest-rerunfailures/issues/10#issuecomment-163636…> > . >

4 8

parametrize and ids
by Brianna Laugher June 6, 2013

June 6, 2013

Hi, I was wondering if anyone has ideas about 'nicer' ways to specify IDs for tests when using the parametrize mark. Example: my parametrize might look like this: @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ]) which results in IDs like this: <Function 'test_noTrendsStartingWithChc[.30-Isol-increasing]'> <Function 'test_noTrendsStartingWithChc[.33-Isol-.35]'> <Function 'test_noTrendsStartingWithChc[.36-Isol-.38]'> <Function 'test_noTrendsStartingWithChc[.39-Isol-clearing]'> <Function 'test_noTrendsStartingWithChc[.42-Isol-.44]'> <Function 'test_noTrendsStartingWithChc[.45-Isol-.47]'> <Function 'test_noTrendsStartingWithChc[.48-Isol-developing]'> <Function 'test_noTrendsStartingWithChc[.51-Isol-.53]'> I can add IDs (I just learned last week! :)) like this: @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ], ids=['IsolRA increasing', 'IsolSH increasing', 'IsolTS increasing', 'IsolRA clearing', 'IsolSH clearing', 'IsolTS clearing', 'IsolRA developing', 'IsolSH developing', 'IsolTS developing', ]) but there are a couple of things I don't like about this: * takes a lot of lines * ID is separate from the test data, bad for maintenance, if the test data gets changed the test name could easily be overlooked when it should be updated too What I would really like is if a string in the tuples could represent the name. Maybe there could be a "magic" fieldname (id, testid, _pytestid ?) that could be stripped out from the data and used as the id. e.g. @py.test.mark.parametrize(('id', 'wx', 'expectedCoverage', 'expectedTrend'), [ ('ChcRA increasing', ['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), ('ChcSH increasing', ['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), ('ChcTS increasing', ['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ]) def test_noTrendsStartingWithChc(wx, expectedCoverage, expectedTrend): # blah This would be really nice. Another alternative could be if you could specify a function that would take the test data and form a string for the test id from it. An example here might be @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), (['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), (['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ], idFn=lambda (wx, cov, trend): '{!s}{} {!s}'.format(wx[0], cov, trend)) However this is quite a bit more messy, and not as flexible as specifying your own string. I would be interested to hear what people think about this, as well as if there are other ways of specifying ids that I have overlooked. (I know I could lay out the data the way I want and use pytest_generate_tests to slice it up for metafunc.parametrize, but I actually prefer explicitly writing out each test case as it tends to be a lot simpler for other developers and future-me to grok.) thanks, Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/

2 3

fixtures as context managers
by holger krekel May 30, 2013

May 30, 2013

Hi all, after having been asked by Andreas Pelme and Vladimir Keleshev, repeatedly, i finally tried to allow fixture functions to be context managers like this: @pytest.fixture def myfixture(): print "setup" yield val print "teardown" With this, you don't need to use "request.addfinalizer" - pytest does this by detecting your yield. I checked this into trunk to allow experimentation - it seems to work fine with scopes and parametrization. For pytest, it's really just a slightly different "fixture function calling" protocol, contained in this commit: https://bitbucket.org/hpk42/pytest/commits/b93ac0cdae02effaa3c136a681cc45bb… Note that i don't plan to extend this yield mechanism to parametrization as that would meanparametrization at test execution time, rather than at collection time. It would be both conceptually and implementation wise hard to do i think. The change is committed to trunk, and you can also do: pip install -i http://pypi.testrun.org -U pytest to get pytest-2.3.6.dev2 at least. feedback welcome, actually without people actively wanting it, it might be thrown out again ;) holger

6 16

fixtures as context managers
by Vladimir Keleshev May 28, 2013

May 28, 2013

2 1

anybody could give pytest/tox training in SF?
by holger krekel May 27, 2013

May 27, 2013

Hi all, I was asked me to give an in-house python testing course in San Francisco. I can probably not make it myself in the next 2-3 months. It's basically the same 3-day course i am giving in Leipzig later in June, see here: http://www.python-academy.com/courses/specialtopics/python_course_testing.h… I am looking for someone to subcontract who could use/modify my existing materials and give the course. If you are interested please contact me privately. thanks, holger

1 0

Question: How to inject an external id to each pytest test case?
by Liao, Shujie May 22, 2013

May 22, 2013

Hi, We want to update the test result automatically from pytest to a web application, we run pytest from Jenkins currently , and our plan is to run a script as post step to make this happen. We have thousands of test cases in an excel file, and now they were all imported to the web tool(the tool generated a unique id for each test), the problem is, the test name in web tool and from pytest are not exactly the same, and so far we don't have plan as well as hands to make them the same.... Our thoughts is to pass the test id to each pytest test case, so that from output result, the id becomes part of test name, then we have the map from pytest test cases to the web tool. We are looking for a feature from pytest, which can help achieve following requirements: 1. Can easily inject an external id to each test case, both non-parameterized and parameterized test. 2. The injected id will be part of test name in output 3. Without impacting existing parameterized mechanism (or need minor modification is fine) For parameterized test, I tried to use @pytest.mark.parametrize(('name'),create_data,ids=['DEMO-1','DEMO-2']), of them, "DEMO-1" and "DEMO-2" are ids from the test cases in web tool, I got this as output: platform linux2 -- Python 2.7.3 -- pytest-2.3.4 collected 2 items <Module 'test_demo.py'> <Class 'TestDemo'> <Instance '()'> <Function 'test_generatekeypair[DEMO-1]'> <Function 'test_generatekeypair[DEMO-2]'> However, for non-parameterized test, seems this solution is less decent, although we can pass the parameter without using it, still seems ugly... Is there a better way we can do this? The information contained in this electronic mail transmission may be privileged and confidential, and therefore, protected from disclosure. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer without copying or disclosing it.

3 3

pytest-django
by Mahdi Yusuf May 22, 2013

May 22, 2013

Hi: I am having issues running pytest-django without inhertiting form Django Testcase, I am also pretty sure this is what is causing the issue to do deal with the database being recreated every time. I would like to reuse my database between test suite runs at least while in development. Thoughts? I am new to using pytest, so any ignorance here in inadvertent. -- Mahdi Yusuf

2 1

Re: [pytest-dev] fixtures as context managers
by holger krekel May 10, 2013

May 10, 2013

On Thu, May 09, 2013 at 17:17 +0200, Ronny Pfannschmidt wrote: > >>im -1 on the flag > > > >Why? And wh is introducing a new "@fixturemanager" decorator which > >duplicates the API better, in your oppinion? > > while the same api, they have different behaviours, > im really not a fan of creating boolean flags to trigger behaviours, > i'd rather see a new api that allows more kinds of fixture > setups/teardowns/parametrizations to be implemented and implementing > the ones for function/contextmanager fixtures on top of that, > so that we get a more flexible base and clear separation of behaviour Not sure what you are referring to here. API implementation layers or exposed APIs? Let's also see what Andreas and Bruno have to say on this api design topic. cheers, holger > -- Ronny >

3 2