[pytest-dev] improve -k/-m exercise / want to help pytest?

[holger krekel]

On Wed, Oct 22, 2014 at 10:20 +0200, Dinu Gherman wrote:
> holger krekel:
> 
> > Hi all,
> > 
> > is anyone interested in a little exercise that would improve pytest usage?
> > It doesn't require deep pytest knowledge.
> > 
> > This is about the "-k" and "-m" selection mini-language, 
> > namely expressions like:
> > 
> >    marker1 and marker2
> >    not marker1 and (marker2 or marker3)
> > 
> > and so forth.  The thing is that pytest currently uses Python's "eval" with
> > a custom dictionary which means that names such as "marker1"
> > cannot have dots or other punctuation and also other things are not
> > possible like running all tests that don't have "4.3" (coming from a
> > parametrized test) in it:
> > 
> >    -k 'not 4.3'
> > 
> > This would currently select all tests because "4.3" is true.  Various
> > people have run into traps like this.
> > 
> > So what is needed here are tests and code that, given a set of names
> > and an expression string, determines if the expression is true or not.
> > And everything except "and" and "or" are valid words. To use the latter
> > two words probably need to have some escaping like "\and" or so but
> > that's a bonus task :)
> > 
> > Anyone up for this little exercise?  Or other comments?
> 
> Not sure I have the time, but still a few comments:
> 
> - I like the planned feature.
>
good :)

> - It would be nice to have a dotted syntax allowing to specify a specific test method in a specific test class, something like -k TestClassX.test_method_y (like it is possible with unittest).

Hum, that wouldn't directly work, i am afraid, unless we put
the full dotted path of test items into the keywords (which we could i guess)

> - Is pyparsing an option for doing this (maybe a somewhat heavy solution, not sure)?

Nope, external deps are not allowed for this.

> - Or, could the expression be formulated somewhere else, like in conftest.py (in pure Python), as a test selection function, and referenced by the -k option by name? That way these expressions would also be easier to reuse, since on the command-line they simply get lost.

That would require a new option, something like "-g" (selected group) or so
and is somewhat unrelated to the task at hand i think.

cheers,
holger