[Python-Dev] Purpose of Doctests [Was: Best practices for Enum]

Raymond Hettinger raymond.hettinger at gmail.com
Sun May 19 08:41:59 CEST 2013

On May 14, 2013, at 9:39 AM, Gregory P. Smith <greg at krypto.org> wrote:

> Bad: doctests.

I'm hoping that core developers don't get caught-up in the "doctests are bad meme".

Instead, we should be clear about their primary purpose which is to test
the examples given in docstrings.   In many cases, there is a great deal 
of benefit to docstrings that have worked-out examples (see the docstrings
in the decimal module for example).  In such cases it is also worthwhile 
to make sure those examples continue to match reality. Doctests are
a vehicle for such assurance.  In other words, doctests have a perfectly
legitimate use case.

We should continue to encourage users to make thorough unit tests
and to leave doctests for documentation.  That said, it should be
recognized that some testing is better than no testing.  And doctests
may be attractive in that regard because it is almost effortless to
cut-and-paste a snippet from the interactive prompt.  That isn't a
best practice, but it isn't a worst practice either.

Another meme that I hope dispel is the notion that the core developers
are free to break user code (such as doctests) if they believe the
users aren't coding in accordance with best practices.   Our goal is to
improve their lives with our modifications, not to make their lives 
more difficult.

Currently, we face an adoption problem with Python 3.  At PyCon,
an audience of nearly 2500 people said they had tried Python 3 
but weren't planning to convert to it in production code.  All of the
coredevs are working to make Python 3 more attractive than Python 2,
but we also have to be careful to not introduce obstacles to conversion.
Breaking tests makes it much harder to convert (especially because
people need to rely on their tests to see if the conversion was successful).


P.S.  Breaking doctests should also be seen as a "canary in a coal mine."
When they break, it also means that printed examples are out of date,
that code parsers may break, that diffs start being different, that programs
that feed into other programs (perhaps via pipes and filters) may be changing
their interface, etc.     Occasionally, we make need to break such things but
there should be a compelling offsetting benefit (i.e. evaluating thoughtfully
whether "I'm trying to help you by making your constant integers have a
nicer repr" is worth "Sorry, I broke your tests, made your published examples
out of date, and slowed down your code."   -- in some modules it will be worth it,
but in others we should value stability over micro-improvments).

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130518/7a144025/attachment-0001.html>

More information about the Python-Dev mailing list