[issue12947] Examples in library/doctest.html lack the flags
New submission from Éric Araujo <merwok@netwok.org>: The docs for the doctest module contains examples that use doctest flags to control behavior, but the HTML version does not contain the flags, making the examples useless. I don’t know if this is a bug with the HTML builder or something we can fix in markup. ---------- assignee: docs@python components: Documentation messages: 143775 nosy: docs@python, eric.araujo, georg.brandl priority: normal severity: normal status: open title: Examples in library/doctest.html lack the flags versions: Python 2.7, Python 3.2, Python 3.3 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Changes by Ezio Melotti <ezio.melotti@gmail.com>: ---------- nosy: +ezio.melotti _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Ezio prompted me to give precise examples, and I can’t find any in the docs online. Maybe it was only a local build with certain options that caused this problem, I’ll reopen if I find out. ---------- resolution: -> invalid stage: -> committed/rejected status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Georg Brandl <georg@python.org> added the comment: Hehe... Sphinx makes a point of *removing* doctest flags, to enable doctesting of code snippets without distracting the reader with the test-internal flags. I think it's because you used a newer version locally. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Stefano Taschini <taschini@ieee.org> added the comment: Concrete examples can be seen in the section http://docs.python.org/library/doctest.html#option-flags-and-directives For instance at http://docs.python.org/library/doctest.html#doctest.IGNORE_EXCEPTION_DETAIL The doctest flags present in the sources in http://docs.python.org/_sources/library/doctest.txt are all stripped. ---------- nosy: +taschini _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: Thank you. I think it’s clear that for the docs of the doctest flags we need to display snippets with the flags. ---------- resolution: invalid -> stage: committed/rejected -> test needed status: closed -> open _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Stefano Taschini <taschini@ieee.org> added the comment: As far as I can see, Sphinx has a global setting for trim_doctest_flags but lacks the possibility of locally disabling the trimming. A quick workaround would be to have the following sphinx extension added: class ProxyLexer(object): def __init__(self, underlying): self.__underlying = underlying def __getattr__(self, attr): return getattr(self.__underlying, attr) def setup(app): from sphinx.highlighting import lexers if lexers is not None: lexers['pycon-literal'] = ProxyLexer(lexers['pycon']) lexers['pycon3-literal'] = ProxyLexer(lexers['pycon3']) That would allow blocks marked as .. code-block:: pycon-literal or preceded by .. highlight:: pycon-literal to escape the trimming of doctest flags. If that's of any interest I can submit a patch. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Ezio Melotti <ezio.melotti@gmail.com> added the comment: Is there a way to add a :keep-doctest-flags: options to literal blocks? ---------- stage: test needed -> needs patch type: -> enhancement _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Stefano Taschini <taschini@ieee.org> added the comment: Ezio, the patch I attached goes into that direction, by adding a ":trim-doctest-flags: disable" option to the code blocks. I thought I had a good reason for having the option worded as ":trim-doctest-flags: disable" instead of ":keep-doctest-flags:", now I'm not so sure. Note: the patch is against the 2.7 branch. ---------- keywords: +patch Added file: http://bugs.python.org/file25424/issue12947_v0.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Changes by Devin Jeanpierre <jeanpierreda@gmail.com>: ---------- title: Examples in library/doctest.html lack the flags -> doctest directive examples in library/doctest.html lack the flags _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Changes by Sandro Tosi <sandro.tosi@gmail.com>: ---------- nosy: +sandro.tosi _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: The directives should normally be stripped, but not when they are intentionally given to teach their existence, syntax, and use, as in the doctest doc on directives. I opened (and closed -- am trying to anyway) a duplicate, #14865. The problem of directive stripping started in 3.2.0 and subsequently 2.7.3 (after 2.7.2 in June 2011) and contiues in 3.3.0. Sandro Tosi noted [1] https://bitbucket.org/birkenfeld/sphinx/issue/169/strip-doctest-csomments-in... [2] https://bitbucket.org/birkenfeld/sphinx/changeset/d91bf8e465ef Issue and commit from June 2009. My question is whether the added true-by-default ``trim_doctest_flags`` config value can just be added to doctest.rst? That would seem to be the point of having a settable config value, but I am ignorant of how sphinx works. I see that the patch does that, but also a lot more. In any case, this is a nasty regression in the docs and should be fixed somehow before the next releases. It makes the examples actively confusing. ---------- nosy: +terry.reedy _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: I discovered this today as well while reading the doctest documentation. One thing that I never noticed before (and that doesn't seem to be reflected in the comments above) is that many of the code snippet rectangles in the doctest documentation have a small rectangle in the upper-right corner with the text ">>>". If you click on one of these small rectangles, most of the text inside the code snippet rectangle disappears. I haven't confirmed this, but I suspect that it is the code snippets with a doctest directive that have this clickable corner. Is this a feature? What is it for? :) ---------- nosy: +cjerdonek _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Ezio Melotti added the comment:
Is this a feature? What is it for? :)
The >>> is added to every example that contains an interactive session to hide the '>>>' and '...' prompts and the output and make the code copy/pastable, and it's not specific to doctests. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 662fb4bd5f84 by Nick Coghlan in branch '2.7': Issue #12947: Add a note to doctest until the example rendering is fixed http://hg.python.org/cpython/rev/662fb4bd5f84 ---------- nosy: +python-dev _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 679b3e3aadae by Nick Coghlan in branch '3.3': Issue #12947: Add a note to doctest until the example rendering is fixed http://hg.python.org/cpython/rev/679b3e3aadae ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Nick Coghlan added the comment: Since the status quo is thoroughly confusing for readers, I added an explicit note before the affected examples. That note should be removed once the rendering problem is fixed. (The note is there on trunk as well, I just forgot to include the issue number in the merge commit) ---------- nosy: +ncoghlan _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: I thought of an easy work-around we can use after looking at the changeset Terry referenced above:
[2] https://bitbucket.org/birkenfeld/sphinx/changeset/d91bf8e465ef
At the expense of pretty color highlighting, we can enable Pygments' TextLexer for the affected examples (aka "null" lexer). For example-- .. code-block:: text
raise CustomError('message') #doctest: +IGNORE_EXCEPTION_DETAIL Traceback (most recent call last): CustomError: message
I confirmed locally that this works. I realized this might work because the Sphinx changeset referenced above has this logic: # trim doctest options if wanted if isinstance(lexer, PythonConsoleLexer) and self.trim_doctest_flags: ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 18d927fb8671 by Nick Coghlan in branch '2.7': Issue #12947: Better workaround for the problem with doctest directives being stripped from code examples that are intended to illustrate those directives http://hg.python.org/cpython/rev/18d927fb8671 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset d93a59a0a984 by Nick Coghlan in branch '3.3': Issue #12947: Better workaround for the problem with doctest directives being stripped from code examples that are intended to illustrate those directives http://hg.python.org/cpython/rev/d93a59a0a984 New changeset 26200f535296 by Nick Coghlan in branch 'default': Merge #12947 workaround from 3.3 http://hg.python.org/cpython/rev/26200f535296 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Nick Coghlan added the comment: Adopted Chris's workaround for now. I kept a reworded version of the preceding note (with the link to this bug), so readers know that the lack of syntax highlighting is intended-but-not-desired. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: Thanks, Nick. It looks like there are a few more though? I'm counting four more in default (search for "doctest:"): three IGNORE_EXCEPTION_DETAIL and one ELLIPSIS. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Nick Coghlan added the comment: I only changed the ones that were specifically in the section explaining doctest directives. There are probably others, but it didn't seem necessary to change them and lose the syntax highlighting at this point. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Terry J. Reedy added the comment: http://docs.python.org/py3k/library/doctest.html#option-flags-and-directives has the improved version. Thanks. Here, the complete example text is more important than the hi-lites. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Nick Coghlan added the comment: Ah, I see the other examples Chris is talking about now. Yes, the workaround should be applied in those cases as well. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 00555659253d by Chris Jerdonek in branch '3.3': Issue #12947: Divide doctest "Option Flags and Directives" section into two. http://hg.python.org/cpython/rev/00555659253d New changeset 467c9f663b77 by Chris Jerdonek in branch 'default': Issue #12947: Merge doctest documentation improvements from 3.3. http://hg.python.org/cpython/rev/467c9f663b77 New changeset 15bfa778ec21 by Chris Jerdonek in branch '2.7': Issue #12947: Backport doctest documentation improvements from 3.3. http://hg.python.org/cpython/rev/15bfa778ec21 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment:
Yes, the workaround should be applied in those cases as well.
I added the workaround to the last remaining example. Nick was right before in that three of the four examples I mentioned above were in the part about "option flags" rather than in the part about "directives". I made those two parts into their own sections for added clarity. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 9057da41cf0b by Georg Brandl in branch '3.2': Issue #12947: revert earlier workaround and use a monkey-patch to enable showing doctest directives only in the doctest docs. http://hg.python.org/cpython/rev/9057da41cf0b New changeset 1202c1449d83 by Georg Brandl in branch '3.3': Issue #12947: revert earlier workaround and use a monkey-patch to enable showing doctest directives only in the doctest docs. http://hg.python.org/cpython/rev/1202c1449d83 ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset ffe9f644d5af by Georg Brandl in branch '2.7': Issue #12947: revert earlier workaround and use a monkey-patch to enable showing doctest directives only in the doctest docs. http://hg.python.org/cpython/rev/ffe9f644d5af ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: Thanks, Georg. Can this new directive be applied to individual code blocks though (or be modified to do so)? After a closer review (as mentioned in my comment above), it seems that the doctest directives should only be displayed in the code snippets that are illustrating the use of doctest directives (as opposed to, say, the snippets that are illustrating doctest options without using directives). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Georg Brandl added the comment: I don't see any blocks in doctest.rst where the directives should be stripped, so it's enough to do it per-file. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: Like these that discuss the behavior of the doctest.IGNORE_EXCEPTION_DETAIL option (but before doctest directives have been introduced): http://hg.python.org/cpython/file/0d6910bef979/Doc/library/doctest.rst#l543 The key is that you can have options take effect without using directives, so it would be good if people can see examples like that. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: Georg suggested removing the doctest directives from the examples I mentioned. As currently written, they don't help them pass if doctest were run on the .rst file. That sounds fine to me for now. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Roundup Robot added the comment: New changeset 705a70b0fff4 by Chris Jerdonek in branch '3.3': Issue #12947: Remove doctest directives from the doctest examples in the "Option Flag" section. http://hg.python.org/cpython/rev/705a70b0fff4 New changeset 9475fc82768e by Chris Jerdonek in branch 'default': Issue #12947: Merge doctest documentation change from 3.3. http://hg.python.org/cpython/rev/9475fc82768e New changeset 8704f9e7ad7c by Chris Jerdonek in branch '2.7': Issue #12947: Backport doctest documentation change from 3.3. http://hg.python.org/cpython/rev/8704f9e7ad7c ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Chris Jerdonek added the comment: This seems to look good now. Can this be closed, or is there something else that needed to be done as well? ---------- resolution: -> fixed stage: needs patch -> commit review status: open -> pending _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
Terry J. Reedy added the comment: I looked at http://docs.python.org/py3k/library/doctest.html#directives and the examples with directives looked fine to me, with directive, syntax markup, and 'hide prompt' box. I also checked the next section and saw the one example that needed to have the directive visible. Really nice. ---------- stage: commit review -> committed/rejected status: pending -> closed _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue12947> _______________________________________
participants (10)
-
Chris Jerdonek -
Devin Jeanpierre
-
Ezio Melotti
-
Georg Brandl
-
Nick Coghlan
-
Roundup Robot
-
Sandro Tosi
-
Stefano Taschini
-
Terry J. Reedy
-
Éric Araujo