<br><br><div class="gmail_quote">On 27 February 2012 20:35, Michael Foord <span dir="ltr"><<a href="mailto:fuzzyman@gmail.com">fuzzyman@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><br><div class="gmail_quote"><div class="im">On 18 February 2012 04:24, Ian Bicking <span dir="ltr"><<a href="mailto:ianb@colorstudy.com" target="_blank">ianb@colorstudy.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p></p><div>On Feb 17, 2012 4:12 PM, "Nick Coghlan" <<a href="mailto:ncoghlan@gmail.com" target="_blank">ncoghlan@gmail.com</a>> wrote:<br>
><br>
> On Sat, Feb 18, 2012 at 7:57 AM, Mark Janssen <<a href="mailto:dreamingforward@gmail.com" target="_blank">dreamingforward@gmail.com</a>> wrote:<br></div><div>
> > Anyway... of course patches welcome, yes... ;^)<br>
><br></div><div>
> Not really. doctest is for *testing code example in docs*. If you try<br>
> to use it for more than that, it's likely to drive you up the wall, so<br>
> proposals to make it more than it is usually don't get a great<br>
> reception (docs patches to make it's limitations clearer are generally<br>
> welcome, though). The stdib solution for test driven development is<br>
> unittest (the vast majority of our own regression suite is written<br>
> that way - only a small proportion uses doctest).</div><p></p>
<p>This pessimistic attitude is why doctest is challenging to work with at times, not anything to do with doctest's actual model. The constant criticisms of doctest keep contributors away, and keep its many resolvable problems from being resolved. </p>
</blockquote></div><div><br>Personally I think there are several fundamental problems with doctest *as a unit testing tool*. doctest is *awesome* for testing documentation examples but in particular this one:<br><br>* Every line becomes an assertion - in a unit test you typically follow the arrange -> act -> assert pattern. Only the results of the *assertion* are relevant to the test. (Obviously unexpected exceptions at any stage are relevant....). With doctest you have to take care to ensure that the exact output of *every line* of your arrange and act steps also match, even if they are irrelevant to your assertion. (The arrange and act steps will often include lines where you are creating state, and their output is irrelevant so long as they put the right things in place.)<br>
<br>The particular implementation of doctest means that there are additional, potentially resolvable problems that are also a damn nuisance in a unit testing fail:<br></div></div></blockquote><div><br></div><div>Jeepers, I changed direction mid-sentence there. It should have read something along the lines of:</div>
<div><br></div><div>As well as fundamental problems, the particular implementation of doctest suffers from these potentially resolvable problems:</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div><br>Execution of an individual testing section continues after a failure. So a single failure results in the *reporting* of potentially many failures.<br>
<br>The problem of being dependent on order of unorderable types (actually very difficult to solve).<br><br>Things like shared fixtures and mocking become *harder* (although by no means impossible) in a doctest environment.<br>
<br>Another thing I dislike is that it encourages a "test last" approach, as by far the easiest way of generating doctests is to copy and paste from the interactive interpreter. The alternative is lots of annoying typing of '>>>' and '...', and as you're editing text and not code IDE support tends to be worse (although this is a tooling issue and not a problem with doctest itself).<br>
<br></div></div></blockquote><div><br></div><div><br></div><div>More fundamental-ish problems:</div><div><br></div><div> Putting debugging prints into a function can break a myriad of tests (because they're output based).</div>
<div><br></div><div> With multiple doctest blocks in a test file running an individual test can be difficult (impossible?).</div><div><br></div><div> I may be misremembering, but I think debugging support is also problematic because of the stdout redirection.</div>
<div><br></div><div>So yeah. Not a huge fan.</div><div><br></div><div>All the best,</div><div><br></div><div>Michael </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><div>So whilst I'm not against improving doctest, I don't promote it as a unit testing tool and disagree that it is suited to that task.<br><br>All the best,<br><br>Michael Foord<br><br><br>
</div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="im">
<div>
<p>> An interesting third party alternative that has been created recently<br>
> is behave: <a href="http://crate.io/packages/behave/" target="_blank">http://crate.io/packages/behave/</a></p>
</div><p>This style of test is why it's so sad that doctest is ignored and unmaintained. It's based on testing patterns developed by people who care to promote what they are doing, but I'm of the strong opinion that they are inferior to doctest.</p>
<span><font color="#888888">
<p> Ian<br></p>
</font></span><br></div><div class="im">_______________________________________________<br>
Python-ideas mailing list<br>
<a href="mailto:Python-ideas@python.org" target="_blank">Python-ideas@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/python-ideas" target="_blank">http://mail.python.org/mailman/listinfo/python-ideas</a><br>
<br></div></blockquote></div><span class="HOEnZb"><font color="#888888"><br><br clear="all"><br>-- <br><pre cols="72"><a href="http://www.voidspace.org.uk/" target="_blank">http://www.voidspace.org.uk/</a><br><br>May you do good and not evil<br>
May you find forgiveness for yourself and forgive others<br>
May you share freely, never taking more than you give.<br>-- the sqlite blessing <a href="http://www.sqlite.org/different.html" target="_blank">http://www.sqlite.org/different.html</a></pre>
<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><pre cols="72"><a href="http://www.voidspace.org.uk/" target="_blank">http://www.voidspace.org.uk/</a><br><br>May you do good and not evil<br>May you find forgiveness for yourself and forgive others<br>
May you share freely, never taking more than you give.<br>-- the sqlite blessing <a href="http://www.sqlite.org/different.html" target="_blank">http://www.sqlite.org/different.html</a></pre>
<br>