[Python-Dev] Code coverage reporting.

[Titus Brown]

On Mon, Jun 19, 2006 at 08:37:30AM -0400, Benji York wrote:
-> Brett Cannon wrote:
-> >But it does seem accurate; random checking of some modules that got high 
-> >but not perfect covereage all seem to be instances where dependency 
-> >injection would be required to get the tests to work since they were 
-> >based on platform-specific things.
-> 
-> >I don't know if we need it hooked into the buildbots (unless it is dirt 
-> >cheap to generate the report).
-> 
-> It would be interesting to combine the coverage over several platforms 
-> and report that.

Yes, I noticed that the platform specific stuff doesn't get covered, of
course.  It's very easy to do, *if* there's any way to get the coverage
database from a central location (or send it back to a central location).

It might be interesting to run coverage analysis -- either figleaf or
Ned Batchelder's module[0] -- once a week on select buildbot machines
(one linux, one windows, one mac, or some such) and make the coverage
databases available via something like a downloadable static file.  Then
anyone could download those files and do Interesting Things with them.

--titus

[0] I'm sorry, I don't know how Walter Dorwald generates his coverage;
if it's OSS, then it'd be better to use because it shows C code coverage
as well.

p.s. Here's the diff for regr_test:

Index: Lib/test/regrtest.py
===================================================================
--- Lib/test/regrtest.py        (revision 46972)
+++ Lib/test/regrtest.py        (working copy)
@@ -1,4 +1,5 @@
 #! /usr/bin/env python
+import figleaf

 """Regression test.

@@ -333,7 +334,11 @@
             tracer.runctx('runtest(test, generate, verbose, quiet, testdir)',
                           globals=globals(), locals=vars())
         else:
+            figleaf.start(False)
             ok = runtest(test, generate, verbose, quiet, testdir, huntrleaks)
+            figleaf.stop()
+            figleaf.write_coverage('.figleaf')
+
             if ok > 0:
                 good.append(test)
             elif ok == 0: