[py-dev] py.test: customizing collector
I got a custom collector working okay; this checks for a function attribute "params", which is a list of tuples, each of which is an argument. I did it like this: from py.test.collect import PyCollector class ParamCollector(PyCollector): def collect_function(self, extpy): if not extpy.check(func=1, basestarts='test_'): return func = extpy.resolve() if hasattr(func, 'params'): params = func.params for param in params: yield self.Item(extpy, *param) else: yield self.Item(extpy) The problem is that all the tests have the same name, at least as displayed. I notice that item.extpy is used as the displayed name for the test; at least some other function should be used (py.test.report.text.summary:192) so that it's easier to override. But I guess it should also fit into --session, and I don't know what that uses for its test ID...? It might be nice if this parameterization was built into the standard collector in some way, as it's a very common case. I'm not sure if it's best as an attribute, a similarly-named object (e.g., test_func and test_func_params), or if there's other ideas. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Ian Bicking wrote:
It might be nice if this parameterization was built into the standard collector in some way, as it's a very common case. I'm not sure if it's best as an attribute, a similarly-named object (e.g., test_func and test_func_params), or if there's other ideas.
Another thought occurred to me here -- one way to do this would be like, say: def add_params(params): """ A decorator, e.g.:: @add_params([(1, 'i'), (2, 'ii'),]) def test_roman(input, output): assert toRoman(input) == output """ def decorator(func): return ParamItemizer(func, params) return decorator class ParamItemizer(Collector): def __init__(self, func, params): self.func = func self.params = params def __iter__(self): for param in params: yield Item(self.func, *param) Of course, it's not nearly that simple because of the extpy stuff when collecting and making items -- Item doesn't take a function, but takes an extpy object, which is introduces a whole new concept into the system. Nor do the collectors seem to look for collectors inside the module, except for a collector that completely overrides the normal collector. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Ian Bicking wrote:
def add_params(params): """ A decorator, e.g.::
@add_params([(1, 'i'), (2, 'ii'),]) def test_roman(input, output): assert toRoman(input) == output """ def decorator(func): return ParamItemizer(func, params) return decorator
Extending this idea, other similar decorators could read the parameters from other data sources, like a spreadsheet or HTML document, similar to Fitnesse (http://www.fitnesse.org/), e.g., @add_html_params('test_document.html', id='test_table1') -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
Hi Ian, [Ian Bicking Sun, Nov 21, 2004 at 02:26:02AM -0600]
I got a custom collector working okay; this checks for a function attribute "params", which is a list of tuples, each of which is an argument. I did it like this:
from py.test.collect import PyCollector
class ParamCollector(PyCollector):
def collect_function(self, extpy): if not extpy.check(func=1, basestarts='test_'): return func = extpy.resolve() if hasattr(func, 'params'): params = func.params for param in params: yield self.Item(extpy, *param) else: yield self.Item(extpy)
The problem is that all the tests have the same name, at least as displayed.
Yes, giving a good name to parametrized tests is a problem. I suppose some more thinking is needed here. I am still at a PyPy sprint and sorting out some business stuff at the same time. So i can't immediately dive deeper here. Note that i have code sitting in a branch that takes a different angle at customizing the collecting process, inspired by your earlier comment that "extpy" introduces a new concept where one just wants to have parametrized tests or have some custom logic to get at or produce test functions. I agree that this should possibly integrate more naturally. Btw, in the branch you can do a simple yield function, arg1, arg2, ... in a test function. However, this doesn't solve the naming (and addressing) problem. more later, holger
holger krekel wrote:
Note that i have code sitting in a branch that takes a different angle at customizing the collecting process, inspired by your earlier comment that "extpy" introduces a new concept where one just wants to have parametrized tests or have some custom logic to get at or produce test functions. I agree that this should possibly integrate more naturally. Btw, in the branch you can do a simple
yield function, arg1, arg2, ...
in a test function. However, this doesn't solve the naming (and addressing) problem.
In DataTest I used the first argument as the ID. There were a couple simple transformations -- get rid of non-alphanumeric characters, truncate the length, then make sure it is unique by adding a number if necessary. When I wanted specific names in my tests, I'd just use the first argument as the name then ignore it in the function. It seems a little hackish, but in practice I don't really see many problems; it's fairly predictable and simple. -- Ian Bicking / ianb@colorstudy.com / http://blog.ianbicking.org
participants (2)
-
hpk@trillke.net -
Ian Bicking