[Python-ideas] Fwd: doctest

[Mark Janssen]

On Fri, Feb 17, 2012 at 5:43 PM, Devin Jeanpierre
<jeanpierreda at gmail.com> wrote...

I, firstly, thank you for your thoughtful reply.  I myself am rather
busy, but totally think it's worth the effort.

> On Fri, Feb 17, 2012 at 4:57 PM, Mark Janssen <dreamingforward at gmail.com> wrote:
>> 1. Execution context determined by outer-scope doctest defintions.
>
> I'm not sure what you mean, but it might be relevant that Sphinx lets
> you define multiple scopes for doctests.

Something like this:  In a class definition doctest, I may put various
definitions ("SetUp" constructs) of interesting and useful class
initializations.  If my class is a Graph, say, I might define several
Graphs (which might produce testable output, or in any case should not
throw an error), which could then be used (as "globals") in the inner
doctests of the various class methods.  It makes no sense to define
them again in each doctest.  The hitch I suppose would be defining the
possible "TearDown' code which would have to be done after the
inner-scope doctests all run.  This would require some syntactical
feature either in doctest or python itself (this latter would be
something like an extra docstring at the end of a class definition,
but such a construct would really only be interesting if test()ing was
built-in to the python interpreter itself (if even then)).  This
really is the only way doctest scoping should make sense.  Any other
way is probably not organizing codetest to docs well.  (In other words
it would enforce a certain testing standard of good practices.)

>I feel like its approach is
> the right one, but it isn't reusable in Python docstrings. That said,
> I think users of doctest have moved away from embedded doctests in
> docstrings -- it encourages doctests to have way too many "examples"
> (test cases), which reduces their usefulness as documentation.

Again, I think this is an example of python not really having
test-driven development built-in.  Complicated doctests are a result
of too coarse of grain in the method definitions, usually (or
probably) a result of other called methods not having their own
doctests, so the slack is being picked-up in an ad-hoc way.  This is
just mostly speculation, I haven't actually gone through any examples.
 I'd be interested in viewing some though if you have them.

>> 2. Smart Comparisons that will detect output of a non-ordered type
>> (dict/set), lift and recast it and do a real comparison.
>
> I think it's better to just always use ast.literal_eval on the output
> as another form of testing for equivalence. This could break code, but
> probably not any code worth caring about.
>
> (in particular,
>    >>> print 'r""'
>    ""

Hmm, I think that would pass in doctest's current framework, which
just tests syntactic characters without regard to semantics.  However,
if one were to fix the dict ordering issue, it would have to gain a
minimum semantic knowledge (like an unordered grouping starts and
terminates with the characters "{" and "}")

>> Anyway... of course patches welcome, yes...  ;^)
>
> Not exactly... doctest has no maintainer, and so no patches ever get
> accepted. If you want to improve it, you'll have to fork it. I hope
> you're that sort of person, because doctest can totally be improved.
> It suffers a lot from people thinking of what it is rather than what
> it could be. :(

I agree!  I'm a bit like yourself though, swamped with other
priorities.   But I'm glad to know about your fork, although it looks
like a efforts are a bit orthogonal to each other....
> This is all assuming your intentions are to contribute rather than
> only suggest. Not that suggestions aren't welcome, I suppose, but
> maybe not here. doctest is not actively developed or maintained
> anywhere, as far as I know. (I want to say "except by me", because
> that'd make me seem all special and so on, but I haven't committed a
> thing in months.)

Well I appreciate your taking the time.  I will make another look at
the code and see what it would take.

> I definitely hope you help to make the doctest world better. I think
> it fills a role that should be filled, and its neglect is unfortunate.

I'm glad someone appreciates it.  I really think the idea should be
integrated more deeply so that it becomes a natural habit for python
programmers.  The test() as a built-in idea came from another doctest
fan.   It would sit right alongside the help() built-in.  Maybe the
idea will gain traction...

mark