Would like to add Edward Loper as a Python developer

Edward Loper runs the Natural Language Toolkit project on SF: http://nltk.sourceforge.net/ and the Epydoc project: http://epydoc.sourceforge.net/ More importantly <wink>, he's a heavy doctest user and has lots of ideas for improving it. doctest's implementation has grown unwieldy, and is in need of refactoring. Jim (Fulton), Edward and I intend to do a lot there over the coming two weeks, and giving Edward commit privileges would simplify and speed the process. This is intended to be limited to doctest changes. Are there objections? If anyone else is interested in contributing to doctest surgery, let me know. A Wiki area might be a decent way to coordinate. Ideas already on the table include restructuring for plugability (e.g., there's no sane way for you to take over doctest error reporting now, but restructuring to do much more via (overridable) class methods is a sane alternative); new gimmicks like notation for "there's a blank line here in this expected output"; a sane way to get into the debugger upon a doctest failure (right now, if you try, doctest is still swallowing stdout, so you can't see what you're doing! Jeremy did some quick hacks around that for Zope3 already); better support for writing "whole file" doctests (tutorial narratives liberally illustrated with Python examples -- Zope3 is finding these very helpful); and documenting all the additions that snuck into Python 2.3.

Edward must not be trying hard enough: nobody hates him <wink>. I got 3 +1's in private, 1 in public, not counting Jim or me, and no objections, so I just added Edward as a Python developer. There's also a new, too-telegraphic Wiki page here: http://zope.org/Members/tim_one/DoctestIdeas Anyone with a zope.org account(*) should be able to add comments there, and/or edit it. You're encouraged. (*) If you don't have one, click the Join link in the upper right corner. It's free, and there are no bad consequences (e.g., you won't even get Zope marketing email as a result -- although I'm not sure whether that's intentional or just because Barry left before finishing ZpamBlaster <heh>).

Tim Peters wrote:
I noticed that the "whole file doctests" item didn't make it to the DoctestIdeas page. I was looking for more details about that. I've found that for tutorial narratives, I really don't want to bother with expected output. For complex software, the output may not always be the same anyway. Also, it's not fun to manipulate input code that has prompts and is within a multi-line string. Syntax highlighting is not available, for one thing. I've made my own tutorial generator. Given this input script: ## Double hash comments treated as meta comments and not output to ## the tutorial. """ Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: """ import os """ Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. """ def foo(): """foo description here""" print 'hello' # be friendly #my_path = 'foo/bar' my_path = os.path.join('foo', 'bar') my_path foo() the output is: Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: >>> import os Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. >>> def foo(): ... """foo description here""" ... print 'hello' # be friendly ... >>> #my_path = 'foo/bar' >>> my_path = os.path.join('foo', 'bar') >>> my_path 'foo/bar' >>> foo() hello -John -- http://giftfile.org/ :: giftfile project

[John Belmonte]
I noticed that the "whole file doctests" item didn't make it to the DoctestIdeas page.
There's more on the wiki now, but no real details. Code for it exists in the doctest.py now on tim-doctest-branch.
I was looking for more details about that.
Here's the class documentation. In context (but not in isolation as here), it's clear that it creates a unittest.TestSuite: def DocFileSuite(*paths, **kw): """Creates a suite of doctest files. One or more text file paths are given as strings. These should use "/" characters to separate path segments. Paths are relative to the directory of the calling module, or relative to the package passed as a keyword argument. A number of options may be provided as keyword arguments: package The name of a Python package. Text-file paths will be interpreted relative to the directory containing this package. The package may be supplied as a package object or as a dotted package name. setUp The name of a set-up function. This is called before running the tests in each file. tearDown The name of a tear-down function. This is called after running the tests in each file. globs A dictionary containing initial global variables for the tests. """ For concrete examples, put the Zope3 source code on a wall, and throw darts at it.
I've found that for tutorial narratives, I really don't want to bother with expected output.
Do you show any output at all? If not, this isn't for you.
For complex software, the output may not always be the same anyway.
We want to focus on invariant (guaranteed) behavior in tests. I think the same is true of good docs. If you can't say anything about the output, you can't test it, and the documentation must be fuzzy too.
Also, it's not fun to manipulate input code that has prompts and is within a multi-line string.
The files DocFileSuite chews on are text files, not necessarily acceptable input to Python (in fact, they never are in practice). So the multi-line string concern doesn't exist in these. Mucking with prompts is indeed a price of entry.
Syntax highlighting is not available, for one thing.
That depends on the editor, but I agree it's at best clumsy to set up.
So if you put your output in a file, you could feed it exactly as-is to DocFileSuite(), and it would create a unittest for you that verified the output in your examples is in fact what it claims to be.

Edward must not be trying hard enough: nobody hates him <wink>. I got 3 +1's in private, 1 in public, not counting Jim or me, and no objections, so I just added Edward as a Python developer. There's also a new, too-telegraphic Wiki page here: http://zope.org/Members/tim_one/DoctestIdeas Anyone with a zope.org account(*) should be able to add comments there, and/or edit it. You're encouraged. (*) If you don't have one, click the Join link in the upper right corner. It's free, and there are no bad consequences (e.g., you won't even get Zope marketing email as a result -- although I'm not sure whether that's intentional or just because Barry left before finishing ZpamBlaster <heh>).

Tim Peters wrote:
I noticed that the "whole file doctests" item didn't make it to the DoctestIdeas page. I was looking for more details about that. I've found that for tutorial narratives, I really don't want to bother with expected output. For complex software, the output may not always be the same anyway. Also, it's not fun to manipulate input code that has prompts and is within a multi-line string. Syntax highlighting is not available, for one thing. I've made my own tutorial generator. Given this input script: ## Double hash comments treated as meta comments and not output to ## the tutorial. """ Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: """ import os """ Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. """ def foo(): """foo description here""" print 'hello' # be friendly #my_path = 'foo/bar' my_path = os.path.join('foo', 'bar') my_path foo() the output is: Triple-quote comments that start in the first column are treated as tutorial text. First we'll import some modules: >>> import os Some code follows. In addition to hash comments, and triple-quote comments not starting in the first column are preserved. Note that code blocks such as functions must end with a blank line, just as is done in the interactive interpreter. >>> def foo(): ... """foo description here""" ... print 'hello' # be friendly ... >>> #my_path = 'foo/bar' >>> my_path = os.path.join('foo', 'bar') >>> my_path 'foo/bar' >>> foo() hello -John -- http://giftfile.org/ :: giftfile project

[John Belmonte]
I noticed that the "whole file doctests" item didn't make it to the DoctestIdeas page.
There's more on the wiki now, but no real details. Code for it exists in the doctest.py now on tim-doctest-branch.
I was looking for more details about that.
Here's the class documentation. In context (but not in isolation as here), it's clear that it creates a unittest.TestSuite: def DocFileSuite(*paths, **kw): """Creates a suite of doctest files. One or more text file paths are given as strings. These should use "/" characters to separate path segments. Paths are relative to the directory of the calling module, or relative to the package passed as a keyword argument. A number of options may be provided as keyword arguments: package The name of a Python package. Text-file paths will be interpreted relative to the directory containing this package. The package may be supplied as a package object or as a dotted package name. setUp The name of a set-up function. This is called before running the tests in each file. tearDown The name of a tear-down function. This is called after running the tests in each file. globs A dictionary containing initial global variables for the tests. """ For concrete examples, put the Zope3 source code on a wall, and throw darts at it.
I've found that for tutorial narratives, I really don't want to bother with expected output.
Do you show any output at all? If not, this isn't for you.
For complex software, the output may not always be the same anyway.
We want to focus on invariant (guaranteed) behavior in tests. I think the same is true of good docs. If you can't say anything about the output, you can't test it, and the documentation must be fuzzy too.
Also, it's not fun to manipulate input code that has prompts and is within a multi-line string.
The files DocFileSuite chews on are text files, not necessarily acceptable input to Python (in fact, they never are in practice). So the multi-line string concern doesn't exist in these. Mucking with prompts is indeed a price of entry.
Syntax highlighting is not available, for one thing.
That depends on the editor, but I agree it's at best clumsy to set up.
So if you put your output in a file, you could feed it exactly as-is to DocFileSuite(), and it would create a unittest for you that verified the output in your examples is in fact what it claims to be.
participants (5)
-
Hye-Shik Chang
-
Jeremy Hylton
-
John Belmonte
-
Tim Peters
-
Tim Peters