[pytest-dev] parametrize + xfail

[holger krekel]

Hi Brianna,

first off a request to you :) 

You filed https://bitbucket.org/hpk42/pytest/issue/279/ and Floris
improved assertion reporting accordingly.  Could you provide testing
and feedback?

On Mon, Apr 22, 2013 at 16:05 +1000, Brianna Laugher wrote:
> Hi again :)
> 
> A common problem I have is that I have a test that is parametrized with
> py.test.mark.parametrize, I discover a bug, I want to add another test case
> for that bug and mark it as xfail.
> 
> I have done something based on
> http://stackoverflow.com/questions/12364801/how-to-mark-some-generated-tests-as-xfail-skip
> for pytest_generate type functions, but it is awkward to do without
> disrupting the existing cases, and somewhat overkill in cases where the
> xfail is likely to be resolved (the bug is fixed) soon.
> 
> With py.test.mark.parametrize, I notice
> https://bitbucket.org/hpk42/pytest/issue/124/allow-individual-parametrized-values-to-be
> 
> I was just thinking about this now and I wonder if it is possible to build
> a decorator like this?
> 
> @py.test.mark.parametrizexfail('duration', 'expectedBrackets'), [
>     (7, [None, None, None, 7]),
>     (19, [None, 7, 6, 6]),
> ])
> @py.test.mark.parametrize(('duration', 'expectedBrackets'), [
>     (24, [6, 6, 6, 6]),
>     (23, [6, 6, 6, 6]),
>     (25, [6, 6, 6, 6]),
> ])
>
> So 5 cases would be fed into the test, with only the first two marked as
> xfail.

I guess this is possible.  I'd probably prefer something like:

 @py.test.mark.parametrize(('duration', 'expectedBrackets'), [
     pytest.mark.xfail((7, [None, None, None, 7])),
     pytest.mark.xfail((19, [None, 7, 6, 6])),
     (24, [6, 6, 6, 6]),
     (23, [6, 6, 6, 6]),
     (25, [6, 6, 6, 6]),

and is a bit easier to switch between xfail and not.  If you like
that as well please open an issue and at best try to come up with
a patch :)

> Also while I'm at it, it could be good for pytest to issue a warning if
> someone uses a mark called parameterize, parametrise or parameterise,
> because I've been caught pondering why a mark wasn't working properly at
> least once :)

Did you try running with "py.test --strict" (which you can make a general
default through "addopts" in a pytest config file)?  It bails out if you
use non-registered markers. 

cheers,
holger

> cheers
> Brianna
> 
> 
> 
> -- 
> They've just been waiting in a mountain for the right moment:
> http://modernthings.org/

> _______________________________________________
> Pytest-dev mailing list
> Pytest-dev at python.org
> http://mail.python.org/mailman/listinfo/pytest-dev