[Python-ideas] doctest

[Devin Jeanpierre]

On Fri, Mar 2, 2012 at 9:57 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> doctest doesn't compare dicts. If you want to compare dicts, write your
> doctest like this:
>
>>>> d = make_dict(...)
>>>> d == {'a': 42, 'b': 23, 'c': None}
> True

Of course doctest doesn't compare dicts; if it did, nobody would be
objecting to its lack of unordered dict comparison.

That aside, your example is not how people do it in the shell, why
should I do it that way in my documentation? Just because doctest
makes me? Pff. My preferred solution is to change doctest to compare
dicts. :)

> doctest compares *strings*. Under no circumstances do I want the default
> doctest comparison to try to be "clever" by guessing when I want strings to
> match using string equality and when I want strings to match using something
> else.

You keep saying what doctest does now, as if that should affect what
it does in the future. :/

By the way, doctest doesn't do that now. ;) With regards to exception
handling, doctest doesn't compare traceback strings to traceback
strings, it compares an exception object to a (badly) parsed
traceback. doctest isn't string comparison everywhere, just most
places. (Of course, it does the comparison by doing a string
comparison on the exception message.)

As it happens, as a result, doctest exceptions are very hard to screw
up (except for SyntaxErrors). The biggest benefit is that you can
copy-paste a traceback, and doctest doesn't care when the stack frames
differ in details (like line numbers, for example). Or you can use
"..." without enabling ELLIPSIS ;)

Not to mention that it lets you use two different forms of exception
header that come up in different versions of Python, and it still
works in other versions of Python without problems.

So many benefits from doctest not trying to be a strict "do what I
say" string comparison! :p

>> Punting it to a user-defined function is nice for _really_ crazy
>> situations, but dicts and sets are not idiosyncratic or in any way
>> exceptional. doctest itself should handle them the way a naive user
>> would expect.
>
>
> No it shouldn't. doctest should handle them the way they actually are.

Yeah, that's what I think. Except I think that they "actually are"
dicts, and you think they're strings. Your opinion doesn't make sense
to me. They are only strings because that's what doctest turned the
dicts into for convenience. There is no reason in particular that it
has to ever turn them into strings at all -- the only thing making
alternatives inconvenient is the syntax for specifying doctests, not
the internal mechanisms of doctest itself. There's nothing holy about
these string comparisons. They are only a means to an end.

Also, could you give something more concrete about why you believe
everything must be based on strings? I couldn't find any reasoning to
that effect in your post. Also keep in mind that I'm not fond of
literal_eval-ing _both_ sides, I'd much rather only the doctest be
eval'd. (In case that affects your answer any.)

-- Devin