[Python-Dev] doctest, unicode repr, and 2to3

[Terry Reedy]

On 3/4/2010 11:11 PM, "Martin v. Löwis" wrote:
> Johan Harjano ran into an interesting problem when trying to run the
> Django test suite under Python 3.1.
>
> Django has doctests of the form
>
>>>> a6.headline
> u'Default headline'
>
> Even when converting the doctest with 2to3, the expected output is
> unmodified. However, in 3.x, the expected output will change (i.e. not
> produce an u"" prefix anymore).
>
> Now, it might be possible to reformulate the test case (e.g. use print()
> instead of relying on repr), however, this is undesirable as a) the test
> should continue to test in 2.x that the result object is a unicode
> string, and b) it makes the test less readable.
>
> I would like to find a solution where this gets automatically corrected,
> e.g. through 2to3, or through changes to doctest, or through changes of
> str.__repr__.
>
> Any proposal appreciated.

What is the easiest thing that works?

If 2to3 can fix string literals and fix code within doc strings, would 
it be difficult to fix expected strings within doc strings?

On the otherhand, as Foord pointed out, the 'u' prefix is something of a 
CPythonism not required by the language def, so an 'ignore leading u on 
expected output' flag would be useful. But 'correcting'

 >>> a,b
u'ah', u'hah'

is too much to expect. Doctest is both useful and fragile because it 
*almost* imitates human testing methods.

If tests like the above were bunched, and my primary goal were to make 
Django tests work, and work now, without changing the style, I would be 
tempted to use something like the following:

def u(s): #import as needed from a testutil module
   if type(s) is unicode: # str in 3.x
     print(s)
   else print('') #or any 'nevermatch' string

 >>> u(a6.headline)
'Default headline'

which both retains the type test and readability. This would be a 
project specific solution, but then, the other comments suggest that 
wanting to combine a type and value test this way seems to be someone 
project specific.

Terry Jan Reedy