fuzzyman at gmail.com
Mon Feb 27 23:59:17 CET 2012
On 27 February 2012 20:35, Michael Foord <fuzzyman at gmail.com> wrote:
> On 18 February 2012 04:24, Ian Bicking <ianb at colorstudy.com> wrote:
>> On Feb 17, 2012 4:12 PM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>> > On Sat, Feb 18, 2012 at 7:57 AM, Mark Janssen <
>> dreamingforward at gmail.com> wrote:
>> > > Anyway... of course patches welcome, yes... ;^)
>> > Not really. doctest is for *testing code example in docs*. If you try
>> > to use it for more than that, it's likely to drive you up the wall, so
>> > proposals to make it more than it is usually don't get a great
>> > reception (docs patches to make it's limitations clearer are generally
>> > welcome, though). The stdib solution for test driven development is
>> > unittest (the vast majority of our own regression suite is written
>> > that way - only a small proportion uses doctest).
>> This pessimistic attitude is why doctest is challenging to work with at
>> times, not anything to do with doctest's actual model. The constant
>> criticisms of doctest keep contributors away, and keep its many resolvable
>> problems from being resolved.
> Personally I think there are several fundamental problems with doctest *as
> a unit testing tool*. doctest is *awesome* for testing documentation
> examples but in particular this one:
> * Every line becomes an assertion - in a unit test you typically follow
> the arrange -> act -> assert pattern. Only the results of the *assertion*
> are relevant to the test. (Obviously unexpected exceptions at any stage are
> relevant....). With doctest you have to take care to ensure that the exact
> output of *every line* of your arrange and act steps also match, even if
> they are irrelevant to your assertion. (The arrange and act steps will
> often include lines where you are creating state, and their output is
> irrelevant so long as they put the right things in place.)
> The particular implementation of doctest means that there are additional,
> potentially resolvable problems that are also a damn nuisance in a unit
> testing fail:
Jeepers, I changed direction mid-sentence there. It should have read
something along the lines of:
As well as fundamental problems, the particular implementation of doctest
suffers from these potentially resolvable problems:
> Execution of an individual testing section continues after a failure. So a
> single failure results in the *reporting* of potentially many failures.
> The problem of being dependent on order of unorderable types (actually
> very difficult to solve).
> Things like shared fixtures and mocking become *harder* (although by no
> means impossible) in a doctest environment.
> Another thing I dislike is that it encourages a "test last" approach, as
> by far the easiest way of generating doctests is to copy and paste from the
> interactive interpreter. The alternative is lots of annoying typing of
> '>>>' and '...', and as you're editing text and not code IDE support tends
> to be worse (although this is a tooling issue and not a problem with
> doctest itself).
More fundamental-ish problems:
Putting debugging prints into a function can break a myriad of tests
(because they're output based).
With multiple doctest blocks in a test file running an individual test
can be difficult (impossible?).
I may be misremembering, but I think debugging support is also
problematic because of the stdout redirection.
So yeah. Not a huge fan.
All the best,
> So whilst I'm not against improving doctest, I don't promote it as a unit
> testing tool and disagree that it is suited to that task.
> All the best,
> Michael Foord
>> > An interesting third party alternative that has been created recently
>> > is behave: http://crate.io/packages/behave/
>> This style of test is why it's so sad that doctest is ignored and
>> unmaintained. It's based on testing patterns developed by people who care
>> to promote what they are doing, but I'm of the strong opinion that they are
>> inferior to doctest.
>> Python-ideas mailing list
>> Python-ideas at python.org
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing http://www.sqlite.org/different.html
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas