[py-dev] Performance tests with py.test

[Bogdan Opanchuk]

Hello,

I am trying to wrap performance tests for my module into py.test. Each
testcase is, basically, a data preparation stage (can take some time)
and the test itself. Is there some way (hooks, plugins) to make
py.test print the value returned from the testcase (and make testcases
time whatever they want)? What I want is just a table like the one
after "py.test -v", but with return values instead of "PASSED". I
googled it and haven't found any solutions.

At the moment I am doing it as following:
1. Overload "pytest_pyfunc_call()" and save returned result into
"pyfuncitem" parameter.
2. Overload "pytest_runtest_makereport()", look for "perf" mark and
somehow output "item.result".
3. I haven't decided about the way to output it yet, because I am
currently trying to find the place where "PASSED" is printed.
But this seems quite clunky and overcomplicated. Is there any better
way to do this? If not for the data preparation, the currently
existing time measuring system would probably be enough for me
(although I'd still miss the ability to print GFLOPS value instead of
seconds), but the data preparation is testcase-specific and I do not
want to move it to "pytest_generate_tests()" hook.

Thank you in advance.