[Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

[Michael Foord]

On 20 May 2013, at 18:26, Mark Janssen <dreamingforward at gmail.com> wrote:

>>> I'm hoping that core developers don't get caught-up in the "doctests are bad
>>> meme".
>>> 
>>> Instead, we should be clear about their primary purpose which is to test
>>> the examples given in docstrings.
>>> In other words, doctests have a perfectly legitimate use case.
>> 
>> But more than just one ;-)  Another great use has nothing to do with
>> docstrings:  using an entire file as "a doctest".   This encourages
>> writing lots of text explaining what you're doing,. with snippets of
>> code interspersed to illustrate that the code really does behave in
>> the ways you've claimed.
> 
> +1, very true.  I think doctest excel in almost every way above
> UnitTests.  I don't understand the popularity of  UnitTests, except
> perhaps for GUI testing which doctest can't handle.  I think people
> just aren't very *imaginative* about how to create good doctests that
> are *also* good documentation.
> 

Doc tests have lots of problems for unit testing.

* Every line is a test with *all* output part of the test - in unit tests you only assert the specific details you're interested in
* Unordered types are a pain with doctest unless you jump through hoops
* Tool support for editing within doctests is *generally* worse
* A failure on one line doesn't halt execution, so you can get many many reported errors from a single failure
* Try adding diagnostic prints and then running your doctests!
* Tools support in terms of test discovery and running individual tests is not as smooth
* Typing >>> and ... all the time is really annoying
* Doctests practically beg you to write your code first and then copy and paste terminal sessions - they're the enemy of TDD
* Failure messages are not over helpful and you lose the benefit of some of the new asserts (and their diagnostic output) in unittest
* Tests with non-linear code paths (branches) are more painful to express in doctests

and so on...

However doctests absolutely rock for testing documentation / docstring examples. So I'm with Raymond on this one.

All the best,

Michael

> That serves two very good purposes in one.  How can you beat that?
> The issues of teardown and setup are fixable and even more beautifully
> solved with doctests -- just use the lexical scoping of the program to
> determine the execution environment for the doctests.
> 
>> picking-your-poison-ly y'rs  - tim
> 
> Cheers,
> 
> Mark
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html