barest beginnings of test coverage results
I compiled the Python interpreter with the flags necessary to get statement and branch coverage information then generated coverage files using the GNU coverage tool, gcov. If you'd like a peek at the coverage files, have a look at http://www.musi-cal.com/~skip/python/Python/dist/src/ The statement/branch coverage information is for one run of executing Lib/test/regrtest.py. Anywhere there is a C source file with coverage info the filename will end in ".c.gcov". I will eventually segregate the coverage files into a parallel tree and not mix them with the build tree. I would also like to adjust the regrtest.py stuff to generate statement coverage for .py files using my trace module: http://www.musi-cal.com/~skip/python/trace.py. For an example of how far this sort of thing can go, poke around in VTK's nightly quality dashboard: http://public.kitware.com/vtk/quality/MostRecentResults/ The VTK gang has been doing this for a couple years and now displays the following information: * pass/fail and faster/slower summaries of their nightly regression tests * code coverage information (using gcc/gcov as I'm doing) * all this on eight different Unix platforms each night Skip
Wow, Skip! Very impressive. But there's soooooo much data... How do we find the gems? Should we just look at one file per day and try to add testcases that get rid of the ##### comments? Is there a way to sort the files based on the number or significance of the ##### comments? --Guido van Rossum (home page: http://dinsdale.python.org/~guido/)
Guido> But there's soooooo much data... How do we find the gems? Guido> Should we just look at one file per day and try to add testcases Guido> that get rid of the ##### comments? Is there a way to sort the Guido> files based on the number or significance of the ##### comments? All I referred you to yesterday (it was late...) was a CVS tree in which I had built the interpreter with the necessary flags, run regrtest.py, then generated coverage files. The next step is to generate one or more pages of coverage summaries. Then you'll be able to browse something like http://www.musi-cal.com/~skip/python/Python/dist/src/lines.html http://www.musi-cal.com/~skip/python/Python/dist/src/branches.html and zero in on those files with lower than desired coverage. Skip
On Sat, Jul 01, 2000 at 08:21:12AM -0500, Guido van Rossum wrote:
Wow, Skip! Very impressive. But there's soooooo much data... How do we find the gems?
http://www.musi-cal.com/~skip/python/Python/dist/src/summary.html The summary seems pretty good to me. Neil
>> Wow, Skip! Very impressive. But there's soooooo much data... How >> do we find the gems? Neil> http://www.musi-cal.com/~skip/python/Python/dist/src/summary.html Neil> The summary seems pretty good to me. ;-) I got to the problem after Guido responded but apparently before you surfed... Skip
Wow, Skip! Very impressive. But there's soooooo much data... How do we find the gems? Should we just look at one file per day and try to add testcases that get rid of the ##### comments? Is there a way to sort the files based on the number or significance of the ##### comments? --Guido van Rossum (home page: http://dinsdale.python.org/~guido/)
Guido> But there's soooooo much data... How do we find the gems? Guido> Should we just look at one file per day and try to add testcases Guido> that get rid of the ##### comments? Is there a way to sort the Guido> files based on the number or significance of the ##### comments? All I referred you to yesterday (it was late...) was a CVS tree in which I had built the interpreter with the necessary flags, run regrtest.py, then generated coverage files. The next step is to generate one or more pages of coverage summaries. Then you'll be able to browse something like http://www.musi-cal.com/~skip/python/Python/dist/src/lines.html http://www.musi-cal.com/~skip/python/Python/dist/src/branches.html and zero in on those files with lower than desired coverage. Skip
On Sat, Jul 01, 2000 at 08:21:12AM -0500, Guido van Rossum wrote:
Wow, Skip! Very impressive. But there's soooooo much data... How do we find the gems?
http://www.musi-cal.com/~skip/python/Python/dist/src/summary.html The summary seems pretty good to me. Neil
>> Wow, Skip! Very impressive. But there's soooooo much data... How >> do we find the gems? Neil> http://www.musi-cal.com/~skip/python/Python/dist/src/summary.html Neil> The summary seems pretty good to me. ;-) I got to the problem after Guido responded but apparently before you surfed... Skip
participants (3)
-
Guido van Rossum -
Neil Schemenauer -
Skip Montanaro