[Python-ideas] doctest

[Mark Janssen]

On Mon, Feb 27, 2012 at 3:59 PM, Michael Foord <fuzzyman at gmail.com> wrote:

>
>
> On 27 February 2012 20:35, Michael Foord <fuzzyman at gmail.com> wrote:
>
>> Personally I think there are several fundamental problems with doctest
>> *as a unit testing tool*. doctest is *awesome* for testing documentation
>> examples but in particular this one:
>>
>> * Every line becomes an assertion - in a unit test you typically follow
>> the arrange -> act -> assert pattern. Only the results of the *assertion*
>> are relevant to the test. (Obviously unexpected exceptions at any stage are
>> relevant....). With doctest you have to take care to ensure that the exact
>> output of *every line* of your arrange and act steps also match, even if
>> they are irrelevant to your assertion. (The arrange and act steps will
>> often include lines where you are creating state, and their output is
>> irrelevant so long as they put the right things in place.)
>>
>> The particular implementation of doctest means that there are additional,
>> potentially resolvable problems that are also a damn nuisance in a unit
>> testing fail:
>>
>
> Jeepers, I changed direction mid-sentence there. It should have read
> something along the lines of:
>
> As well as fundamental problems, the particular implementation of doctest
> suffers from these potentially resolvable problems:
>
>
>> The problem of being dependent on order of unorderable types (actually
>> very difficult to solve).
>>
>> I just thought of something:  isn't the obvious solution is for doctest
to test the type of an expression's output and if it's in the set of
unordered types {set, dict} to run sort() on it?  Then the docstring author
can just put the (sorted) output of what's expected....

Perhaps I'm hashing a dead horse, but I really would like to see this added
to the issue's tracker as a requested feature.  I may code up the patch
myself, but it helps my brain to have it "on the dev stack".

mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20120301/5e0c8cd0/attachment.html>