Howto? : pydoc, testdoc and docstrings with examples involving escaped quotes inside strings

[Pierre Rouleau]

I am trying to document a simple function with docstrings that will be 
tested automatically with testdoc.  The problem is that the function 
processes strings with escaped quotes and when i describe the function 
and its results inside the docstring:

  1- the escaped double quote changes into a double quote.
  2- this change makes the displayed documentation invalid.
  3- this change makes the doctest fail.

Here's the function and the docstring:

# ------------- cstring_of() ------------------
def cstring_of(text) :
    """Extract & return a single C-style string placed inside text.

    The function extracts the first C-style string from the passed text.
    It allows any type of macro before the C string and allows escaped
    double quotes inside the string.
    [Example]
    >>> line = r'"abcd\"efghi\"jklmnop" // a comment'
    >>> cstring_of(line)
    'abcd\\"efghi\\"jklmnop'
    >>> # The function recognizes the Microsoft _T() macro
    >>> # used to wrap strings.
    >>> line = r'   _T("Cstring with escaped quotes: \"....")  // cmt.'
    >>> cstring_of(line)
    'Cstring with escaped quotes: \"....'
    >>> print cstring_of(line)
    Cstring with escaped quotes: \"....
    """
    # '
    cstring = ''
    text = text[text.index('"')+1:]
    parts = text.split('"')
    for part in parts:
       cstring += part
       if part[-1] == '\\' :
          cstring += '"'
       else:
          return cstring
    return cstring

# ------------- end of cstring_of() ------------------


#==== Running the docstring inside a Python shell:
# If i run the doc string manually from the Python shell, i get:

 >>> from utstring import *
 >>> line = r'"abcd\"efghi\"jklmnop" // a comment'
 >>> cstring_of(line)
'abcd\\"efghi\\"jklmnop'
 >>> line = r'   _T("Cstring with escaped quotes: \"....")  // cmt.'
 >>> cstring_of(line)
'Cstring with escaped quotes: \\"....'
 >>> print cstring_of(line)
Cstring with escaped quotes: \"....
 >>>


# This is fine.
#================================================
# Now, if i process the module where this function is stored with pydoc 
# to generate the documentation, the escaped quote is no longer escaped. 
  # Look at 'quotes: "...."'
# The escaped double quote after the colon is now a double quote.

cstring_of(text)
     Extract & return a single C-style string placed inside text.

     The function extracts the first C-style string from the passed text.
     It allows any type of macro before the C string and allows escaped
     double quotes inside the string.
     [Example]
     >>> line = r'"abcd"efghi"jklmnop" // a comment'
     >>> cstring_of(line)
     'abcd\"efghi\"jklmnop'
     >>> # The function recognizes the Microsoft _T() macro
     >>> # used to wrap strings.
     >>> line = r'   _T("Cstring with escaped quotes: "....")  // cmt.'
     >>> cstring_of(line)
     'Cstring with escaped quotes: "....'
     >>> print cstring_of(line)
Cstring with escaped quotes: "....

#================================================
# The problem now, is that the doctest for the docstring will now fail:
#

Running utstring.__doc__
0 of 0 examples failed in utstring.__doc__
Running utstring.cstring_of.__doc__
Trying: line = r'"abcd"efghi"jklmnop" // a comment'
Expecting: nothing
ok
Trying: cstring_of(line)
Expecting: 'abcd\"efghi\"jklmnop'
*****************************************************************
Failure in example: cstring_of(line)
from line #7 of utstring.cstring_of
Expected: 'abcd\"efghi\"jklmnop'
Got: 'abcd'
Trying: line = r'   _T("Cstring with escaped quotes: "....")  // cmt.'
Expecting: nothing
ok
Trying: cstring_of(line)
Expecting: 'Cstring with escaped quotes: "....'
*****************************************************************
Failure in example: cstring_of(line)
from line #12 of utstring.cstring_of
Expected: 'Cstring with escaped quotes: "....'
Got: 'Cstring with escaped quotes: '
Trying: print cstring_of(line)
Expecting: Cstring with escaped quotes: "....
*****************************************************************
Failure in example: print cstring_of(line)
from line #14 of utstring.cstring_of
Expected: Cstring with escaped quotes: "....
Got: Cstring with escaped quotes:
3 of 5 examples failed in utstring.cstring_of.__doc__

#================================================
#================================================

How can I write an example inside the docstring that shows how escaped 
quotes are handled and get it to show properly inside the documents 
generated with pydoc and tested correctly with doctest?


I am using:
Python 2.2 (#28, Dec 21 2001, 12:21:22) [MSC 32 bit (Intel)] on win32



Thanks!

Pierre