[Python-3000] doctests vs. unittests (was Re: pimp; restructuring the standard library)
chrism at plope.com
Fri Jun 29 07:40:39 CEST 2007
On Jun 28, 2007, at 11:46 PM, Barry Warsaw wrote:
> The thing that convinced me was the realization (assisted by my
> colleagues) that doctests are first and foremost documentation.
> They are testable documentation sure, but the unit tests are
> secondary. There's no question that for things like system
> documentation, the narrative that weaves the testable bits together
> in a well written doctest are much more valuable than the tests.
I suspect it would be even more valuable as documentation if it
didn't give good coverage.
> Most unittest based tests have little or no comments, and nothing
> approaching the narrative in a good doctest, so it's clear that
> unittests are tests first and probably not documentation at all.
This probably isn't the place for this discussion but I'll give an
explanation about why I think that's actually a good thing.
I find that I only get good test coverage when I have more test code
for a component than the implementation of the component I'm trying
to test. At least that's been my experience. I haven't been able
to make the tests more succinct while still testing things adequately.
When coverage gets good, "documentation-ness" of tests suffers. You
can get good test coverage with any sort of tests. But once you get
good test coverage, whatever framework you've chose to write them in,
the tests are no longer very good as narrative documentation because
they're littered with bits of fixture code, edge case assertions, etc.
I don't mind doctest at all really (I just use unittest out of
inertia and personal preference, I'd probably just as happy with nose
or whatever). I just don't like when folks advertise the same
doctest as both a comprehensive set of tests and a component's only
source of documentation, because I don't think it's possible for it
to be both at the same time with any sort of quality in both
That said, having testable documentation is a good thing! I'd just
prefer that that documentation did not include lots of fixture noise.
> A good test suite can benefit from both doctests and unittests and
> I don't think unittest will ever go away (nor should it), but in my
> latest work I'm opting more and more for doctests. That Tim Peters
> is a smart guy after all I guess. :)
I miss "uncle Timmy". :-(
More information about the Python-3000