[Patches] [ python-Patches-1429818 ] trace.py needs to know about doctests

[SourceForge.net]

Patches item #1429818, was opened at 2006-02-11 21:22
Message generated for change (Comment added) made by haypo
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1429818&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Library (Lib)
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marius Gedminas (mgedmin)
Assigned to: Nobody/Anonymous (nobody)
Summary: trace.py needs to know about doctests

Initial Comment:
SchoolTool's test runner uses the 'trace' module to
produce unit test coverage reports.  Many unit tests in
SchoolTool are actually doctests.  When the test runner
invokes trace.CoverageResults.write_results, it gets a
lot of printed error messages like this:

  Not printing coverage data for '<doctest
schooltool.app.browser.tests.test_app.doctest_ACLView[60]>':
[Errno 2] No such file or directory: '<doctest
schooltool.app.browser.tests.test_app.doctest_ACLView[60]>'

and a lot of empty files with nimes like

  <doctest schooltool.timetable.browser.cover

appear in the output directory.

The attached patch fixes both problems.

----------------------------------------------------------------------

Comment By: STINNER Victor (haypo)
Date: 2007-02-27 02:45

Message:
Logged In: YES 
user_id=365388
Originator: NO

Zope version of doctest is patched for coverage testing, get:
http://svn.zope.org/zope.testing/trunk/src/zope/testing/doctest.py

The patch to fix Python 2.4 doctest.py:

--- doctest.py.orig     2007-02-27 02:32:05.000000000 +0100
+++ doctest.py  2007-02-27 02:36:31.000000000 +0100
@@ -340,8 +340,19 @@
     """
     def __init__(self, out):
         self.__out = out
+        self.__debugger_used = False
         pdb.Pdb.__init__(self)

+    def set_trace(self):
+        self.__debugger_used = True
+        pdb.Pdb.set_trace(self)
+
+    def set_continue(self):
+        # Calling set_continue unconditionally would break unit test
coverage
+        # reporting, as Bdb.set_continue calls sys.settrace(None).
+        if self.__debugger_used:
+            pdb.Pdb.set_continue(self)
+
     def trace_dispatch(self, *args):
         # Redirect stdout to the given stream.
         save_stdout = sys.stdout

----------------------------------------------------------------------

Comment By: Marius Gedminas (mgedmin)
Date: 2006-02-15 18:06

Message:
Logged In: YES 
user_id=44660

I do not entirely understand your questions, so I'll try to
describe the situation more clearly.

Imagine this code in foo.py:

  import doctest

  def do_something(x):
      """
         >>> do_something(3)
         4
      """
      if x == 0:
           return -1
      return x + 1

  if __name__ == '__main__':
      import doctest
      doctest.testmod()

Now run

  python -m trace --cover --missing --coverdir=output foo.py

trace.py will produce 'foo.cover' that looks like this

      1: import doctest

      1: def do_something(x):
             """
                >>> do_something(3)
                4
             """
      1:     if x == 0:
  >>>>>>          return -1
      1:     return x + 1

      1: if __name__ == '__main__':
      1:     import doctest
      1:     doctest.testmod()

It will also produce an empty file called '<doctest
__main__.cover', and produce a warning on stdout

  Not printing coverage data for '<doctest
__main__.do_something[0]>': [Errno 2] No such file or
directory: '<doctest __main__.do_something[0]>'

With my patch, trace.py will produce a 'foo.cover' file with
the same content, and it will not create empty files nor
produce any warnings.

(I do not particularly care about tracing Python statements
inside doctests themselves.  I would like to keep this bug
report focused, so if anyone wants doctest coverage in
addition to regular Python code coverage, I suggest opening
a new bug report.)

Given this, I think the answers to your questions are

  (1) yes

  (2) what is 'the code that creates a test'?  If you mean
the '>>> do_something(4)' line, then I'm content if it
remains untracked

  (3) whether doctests are in the same file or not is
irrelevant, in my opinion (in SchoolTool most of the
doctests are in separate files); I do not see where they
might get mistaken for code

  There are no test-portions-that-didn't run, so there's no
need for subdirectories.

  It would not be OK if some of the tests/doctests didn't
actually run because of trace.py.  Luckily, that is not the
case.

----------------------------------------------------------------------

Comment By: Jim Jewett (jimjjewett)
Date: 2006-02-15 17:20

Message:
Logged In: YES 
user_id=764593

Do I understand this correctly, then

(1)  doctests are still run, and contribute to coverage

(2)  The code that creates a test should not be measured for 
coverage. ???

(3)  Because doctests are in the same file as the code, 
rather than the other tests, they get mistaken for code 
whose own coverage should be measured.

If so, it seems like the right solution is to add a 
subdirectory for test-portions-that-didn't-run, and put 
doctests there instead of in the main directory.  

Or are you saying that is OK if some of the tests (and some 
of the doctests) don't actually run in full?

-jJ


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1429818&group_id=5470