Hi, I was wondering if anyone has ideas about 'nicer' ways to specify IDs for tests when using the parametrize mark. Example: my parametrize might look like this: @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ]) which results in IDs like this: <Function 'test_noTrendsStartingWithChc[.30-Isol-increasing]'> <Function 'test_noTrendsStartingWithChc[.33-Isol-.35]'> <Function 'test_noTrendsStartingWithChc[.36-Isol-.38]'> <Function 'test_noTrendsStartingWithChc[.39-Isol-clearing]'> <Function 'test_noTrendsStartingWithChc[.42-Isol-.44]'> <Function 'test_noTrendsStartingWithChc[.45-Isol-.47]'> <Function 'test_noTrendsStartingWithChc[.48-Isol-developing]'> <Function 'test_noTrendsStartingWithChc[.51-Isol-.53]'> I can add IDs (I just learned last week! :)) like this: @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ], ids=['IsolRA increasing', 'IsolSH increasing', 'IsolTS increasing', 'IsolRA clearing', 'IsolSH clearing', 'IsolTS clearing', 'IsolRA developing', 'IsolSH developing', 'IsolTS developing', ]) but there are a couple of things I don't like about this: * takes a lot of lines * ID is separate from the test data, bad for maintenance, if the test data gets changed the test name could easily be overlooked when it should be updated too What I would really like is if a string in the tuples could represent the name. Maybe there could be a "magic" fieldname (id, testid, _pytestid ?) that could be stripped out from the data and used as the id. e.g. @py.test.mark.parametrize(('id', 'wx', 'expectedCoverage', 'expectedTrend'), [ ('ChcRA increasing', ['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), ('ChcSH increasing', ['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), ('ChcTS increasing', ['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ]) def test_noTrendsStartingWithChc(wx, expectedCoverage, expectedTrend): # blah This would be really nice. Another alternative could be if you could specify a function that would take the test data and form a string for the test id from it. An example here might be @py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), (['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), (['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ], idFn=lambda (wx, cov, trend): '{!s}{} {!s}'.format(wx[0], cov, trend)) However this is quite a bit more messy, and not as flexible as specifying your own string. I would be interested to hear what people think about this, as well as if there are other ways of specifying ids that I have overlooked. (I know I could lay out the data the way I want and use pytest_generate_tests to slice it up for metafunc.parametrize, but I actually prefer explicitly writing out each test case as it tends to be a lot simpler for other developers and future-me to grok.) thanks, Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/
Another alternative - I would often prefer just doing what metafunc.addcall does if you don't specify an id, which is just make an integer, like so: <Function 'test_filterPrecipOrTS[0]'> <Function 'test_filterPrecipOrTS[1]'> <Function 'test_filterPrecipOrTS[2]'> <Function 'test_filterPrecipOrTS[3]'> <Function 'test_filterPrecipOrTS[4]'> <Function 'test_filterPrecipOrTS[5]'> Maybe it is wrong and should be updated to use idmaker :P But basically, if the id can't be simple and meaningful to me (and with parametrize it is often not), an integer would be less "visual noise" and in the case of the mark parametrize, easy to map back to the specific test data in the case of a failure. Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/
Hi Brianna, I am not using mark.parametrize as much as you do and usually in simpler scenarios. Here are some comments nevertheless. On Wed, May 29, 2013 at 12:16 +1000, Brianna Laugher wrote:
Hi,
I was wondering if anyone has ideas about 'nicer' ways to specify IDs for tests when using the parametrize mark.
Example:
my parametrize might look like this:
@py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ])
which results in IDs like this:
<Function 'test_noTrendsStartingWithChc[.30-Isol-increasing]'> <Function 'test_noTrendsStartingWithChc[.33-Isol-.35]'> <Function 'test_noTrendsStartingWithChc[.36-Isol-.38]'> <Function 'test_noTrendsStartingWithChc[.39-Isol-clearing]'> <Function 'test_noTrendsStartingWithChc[.42-Isol-.44]'> <Function 'test_noTrendsStartingWithChc[.45-Isol-.47]'> <Function 'test_noTrendsStartingWithChc[.48-Isol-developing]'> <Function 'test_noTrendsStartingWithChc[.51-Isol-.53]'>
I can add IDs (I just learned last week! :)) like this:
@py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra2', 'ra2', 'ra4', 'ra4'], 'Isol', None), (['sh2', 'sh2', 'sh4', 'sh4'], 'Isol', 'increasing'), (['ts2', 'ts2', 'ts4', 'ts4'], 'Isol', None), (['ra2', 'ra2', 'ra2', None], 'Isol', None), (['sh2', 'sh2', 'sh2', None], 'Isol', 'clearing'), (['ts2', 'ts2', 'ts2', None], 'Isol', None), ([None, None, 'ra2', 'ra2'], 'Isol', None), ([None, None, 'sh2', 'sh2'], 'Isol', 'developing'), ([None, None, 'ts2', 'ts2'], 'Isol', None), ], ids=['IsolRA increasing', 'IsolSH increasing', 'IsolTS increasing', 'IsolRA clearing', 'IsolSH clearing', 'IsolTS clearing', 'IsolRA developing', 'IsolSH developing', 'IsolTS developing', ])
but there are a couple of things I don't like about this: * takes a lot of lines * ID is separate from the test data, bad for maintenance, if the test data gets changed the test name could easily be overlooked when it should be updated too
agreed. One a sidenote, yesterday i introduced a briefer way to specify argument names. Your example would start like this: @pytest.mark.parametrize("wx,expectedCoverage,expectedTrend", ...) and you can also have spaces after the commas if you prefer.
What I would really like is if a string in the tuples could represent the name. Maybe there could be a "magic" fieldname (id, testid, _pytestid ?) that could be stripped out from the data and used as the id.
e.g.
@py.test.mark.parametrize(('id', 'wx', 'expectedCoverage', 'expectedTrend'), [ ('ChcRA increasing', ['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), ('ChcSH increasing', ['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), ('ChcTS increasing', ['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ]) def test_noTrendsStartingWithChc(wx, expectedCoverage, expectedTrend): # blah
This would be really nice.
Looks better, maybe the special id name should then be spelled "__id" or so to make it more obvious it's not just an argument name.
Another alternative could be if you could specify a function that would take the test data and form a string for the test id from it. An example here might be
@py.test.mark.parametrize(('wx', 'expectedCoverage', 'expectedTrend'), [ (['ra1', 'ra1', 'ra4', 'ra4'], 'Wide', None), (['sh1', 'sh1', 'sh4', 'sh4'], 'Wide', None), (['ts1', 'ts1', 'ts4', 'ts4'], 'Wide', None), ], idFn=lambda (wx, cov, trend): '{!s}{} {!s}'.format(wx[0], cov, trend))
However this is quite a bit more messy, and not as flexible as specifying your own string.
Messyness depends, i'd say. If you have multiple test functions you could aim to re-use the same id-generating function and could then be sure that the IDs are always correctly denoting the parametrizted data.
I would be interested to hear what people think about this, as well as if there are other ways of specifying ids that I have overlooked. (I know I could lay out the data the way I want and use pytest_generate_tests to slice it up for metafunc.parametrize, but I actually prefer explicitly writing out each test case as it tends to be a lot simpler for other developers and future-me to grok.)
I'd recommend to write a wrapper "@myparametrize" which generates a pytest.mark.parametrize() instance in the end. This way you could use and consolidate your API ideas in real life with today's pytest in real life before aiming for pytest inclusion. best, holger
thanks, Brianna
-- They've just been waiting in a mountain for the right moment: http://modernthings.org/
_______________________________________________ Pytest-dev mailing list Pytest-dev@python.org http://mail.python.org/mailman/listinfo/pytest-dev
On 29 May 2013 17:48, holger krekel <holger@merlinux.eu> wrote:
agreed. One a sidenote, yesterday i introduced a briefer way to specify argument names. Your example would start like this:
@pytest.mark.parametrize("wx,expectedCoverage,expectedTrend", ...)
and you can also have spaces after the commas if you prefer.
I noticed that :) looks good. I think of parametrize like specifying namedtuples so it is a good improvement.
I'd recommend to write a wrapper "@myparametrize" which generates a pytest.mark.parametrize() instance in the end. This way you could use and consolidate your API ideas in real life with today's pytest in real life before aiming for pytest inclusion.
Great idea. I discussed my idea with my workmate and he immediately suggested I use a dictionary, with the key being the test name and the value being the argvalue (test data). In doing that the test order is not guaranteed, but one could use an OrderedDict I suppose if that was important. I have put up a working example here for anyone who is interested: https://gist.github.com/pfctdayelise/5719730 cheers Brianna -- They've just been waiting in a mountain for the right moment: http://modernthings.org/
participants (2)
-
Brianna Laugher -
holger krekel