[Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

[Barry Warsaw]

On May 19, 2013, at 07:28 PM, Tim Peters wrote:

>But more than just one ;-)  Another great use has nothing to do with
>docstrings:  using an entire file as "a doctest".   This encourages
>writing lots of text explaining what you're doing,. with snippets of
>code interspersed to illustrate that the code really does behave in
>the ways you've claimed.

Agreed!  I love separate-file doctests, and the marriage of doctests and
reST/Sphinx is just fantastic.  It's a pleasure to write usage documentation
that contains code samples that are guaranteed to work, because they pass
their doctest.  (I personally don't like long-winded docstring doctests
because they are harder to edit and distract from the code, but YMMV.)

Several years ago, I spent some time experimenting with using doctest for
*everything*.  I deliberately wanted to go that extreme in order to better
explore where doctests are good and where they're not so good.  A general rule
of thumb I came up with is that reST-style doctests are great for explanations
involving mostly good-path usage of a library, or IOW "this is how you're
supposed to use this API, and see it works!".

IME, doctests are not so good at testing all the failure modes, odd corner
cases, and the perhaps less-common good-path use cases.  Fortunately, we have
another useful tool for testing that stuff <wink>.

>I'd rather encourage users to turn their brains on when writing
>doctest files - and when writing unit tests.  I've lost count of how
>many times I've seen a unit test fail, then stared helplessly at the
>unit test code just trying to figure out what the author thought they
>were doing.  A lot of comments in the test code could have helped
>that, but - outside of doctest-based tests - there's typically very
>little explanatory text in testing code.

+1

A rule-of-thumb I use is what I call the FORTH rule[1].  If you should be able
to understand what your own test is trying to accomplish a week later,
otherwise you're not writing very good tests. ;)

-Barry

[1] or PERL rule maybe, depending on the unit of time. :)