[issue36675] Doctest directives and comments not visible or missing from code samples
New submission from Steven D'Aprano <steve+python@pearwood.info>: (Apologies if this is the wrong place for reporting website bugs.) The website is not rendering doctest directives or comments, either that or the comments have been stripped from the examples. On the doctest page itself, all the comments are missing: https://docs.python.org/3/library/doctest.html#directives The first example says: >>> print(list(range(20))) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] but without the directive, the test would fail. Screen shot attached. Doctest directives are also missing from here: https://docs.python.org/3/library/ctypes.html My browser: Firefox 45.1.1 Also checked with text browser "lynx". ---------- assignee: docs@python components: Documentation files: missing_directives.png messages: 340570 nosy: docs@python, steven.daprano priority: normal severity: normal status: open title: Doctest directives and comments not visible or missing from code samples type: behavior Added file: https://bugs.python.org/file48277/missing_directives.png _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Change by Karthikeyan Singaravelan <tir.karthi@gmail.com>: ---------- nosy: +mdk _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: I verified that the line in Doc/library/doctest.rst has the comment. " For example, this test passes::
print(list(range(20))) # doctest: +NORMALIZE_WHITESPACE [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] " However, the comment is omitted from the .html built on Windows by Sphinx 1.8.1. " <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">)))</span> ***html for comment should be here*** <span class="go">[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,</span> <span class="go">10, 11, 12, 13, 14, 15, 16, 17, 18, 19]</span> </pre></div> </div> " To me, this is a bug with building the .html for doctest.rst. Comments are preserved elsewhere. For instance, the code example for https://docs.python.org/3/library/functions.html#dir. The .rst file has " The resulting list is sorted alphabetically. For example:
>>> import struct >>> dir() # show the names in the module namespace # doctest: +SKIP " Both use 3-space colon + indents to mean 'example block'. The only difference is '::' versus ':'. https://devguide.python.org/documenting/#source-code says :: is required. idle.rst also has a code example with comments displayed (I just submitted a PR to not suppress color highlighting.) I leave it to the doc experts to discover why the comments are not included in doctest.html. ---------- nosy: +eric.araujo, ezio.melotti, terry.reedy, willingc stage: -> needs patch title: Doctest directives and comments not visible or missing from code samples -> Doctest directives and comments missing from code samples versions: +Python 2.7, Python 3.7, Python 3.8 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Change by Roundup Robot <devnull@psf.upfronthosting.co.za>: ---------- keywords: +patch pull_requests: +13038 stage: needs patch -> patch review _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Jeroen Demeyer <J.Demeyer@UGent.be> added the comment: Isn't it a *feature* that those doctest directives are not shown? Those directives are meant for the doctest module only, not for the reader of the rendered documentation. ---------- nosy: +jdemeyer _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Éric Araujo <merwok@netwok.org> added the comment: OP is about the documentation page for doctest itself! ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives. The patch only exposes them for doctest.html and not for ctypes or anywhere else. They really should not be in the dir example code that I linked to. https://docs.python.org/3/library/functions.html#dir The problem there are the double comments with both a real comment and a directive. https://devguide.python.org/documenting/#source-code does not say anything about '::' causing suppression of comments and ':' leaving them. It is misleading in implying the '::' is required for a code block. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Jeroen Demeyer <J.Demeyer@UGent.be> added the comment:
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.
Thanks for clarifying. I missed that. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Steven D'Aprano <steve+python@pearwood.info> added the comment:
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives. The patch only exposes them for doctest.html and not for ctypes or anywhere else.
Thanks for the patch, and the extra information, but I disagree with the decision to suppress the directives. The reason I found this problem in the first case was that I started with the ctypes documentation, where it says: "Since some code samples behave differently under Linux, Windows, or Mac OS X, they contain doctest directives in comments." and I was very keen to see those directives so I could learn the right way to deal with platform-dependent doctests. I was very confused that they weren't visible. https://docs.python.org/3/library/ctypes.html I think that doctest directives are as much a part of documenting correct usage as any other part of the example code, and they are (semi-)human readable and (almost) self-documenting. Consider this example from ctypes: >>> c_wchar_p("Hello, World") c_wchar_p(140018365411392) In the absence of a directive, but knowing that it may have been surpressed, I don't know how to interpret that. Is the output some arbitrarily chosen value that doctest ought to skip? Or is that the actual output that c_wchar_p("Hello, World") will return every single time without fail? If I was a ctypes expert, it might be blindingly obvious to me, but I'm not, so I'm left in the dark. I don't know whether I should expect that precise output each and every time, or something platform and implementation specific. If the directive #doctest:+SKIP was visible, I would know that it was an arbitrarily chosen example. My preference would be: - keep the doctest directives visible, everywhere; - make them a clickable link to the appropriate section in the doctest documentation; - and, if possible, on mouse-over, they should display a tooltip with a message like "The output of this example is arbitrary." Or similar wording.
They really should not be in the dir example code that I linked to. https://docs.python.org/3/library/functions.html#dir
On the contrary: I think that the presence of the +SKIP directive helps demonstrate that the output shown is a made-up example, not normative. (Of course it helps that I know doctest, but even if I didn't, the tooltip message would help.) ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Cheryl Sabella <cheryl.sabella@gmail.com> added the comment: ;tldr; There is a global configuration flag to show all the doctest directives for all the docs that are built. The default is to suppress the doctest directives. If a global setting isn't desired, then each doctest example will have to be changed individually. I've looked at the Sphinx documentation for this and there are a few points to reference. * `::` defines a Literal block in reST. [1] A literal block doesn't do anything special except from the last paragraph is this section of the docs: "Code highlighting can be enabled for these literal blocks on a document-wide basis using the `highlight` directive and on a project-wide basis using the `highlight_language` configuration option. The code-block directive can be used to set highlighting on a block-by-block basis. These directives are discussed later." CPython has the `highlight-language` set to python3. * A doctest block does not need to be in a Literal Block. [2] They are identified automatically. [3] According to the doc, doctest directives are suppressed from presentation: Also, you can give inline doctest options, like in doctest: >>> datetime.date.now() # doctest: +SKIP datetime.date(2008, 1, 1) They will be respected when the test is run, but stripped from presentation output. * Any section (doesn't need the single colon (`:`) can have the `code-block` directive on it. [4] The `code-block` directive can define the language for highlighting. Note that in the Sphinx doctest [4] documentation, there is a config option `trim_doctest_flags`, which is True by default. Setting this to False will show all the doctest directives for all the doctests in the documentation. The nice thing about the doc build recognizing a section as a doctest is that it adds the toggle in the upper right of the block to 'Hide the toggle and output' so that the example can be copy and pasted more easily. Changing it into something other than a doctest (like using a `code-block` directive, takes away this option. Using `trim_doctest_flags=False` retains the hide button. Options: 1. Change global setting to false to show all doctest directives. 2. Change only the blocks where the doctest directives need to be shown, but this makes them lose the 'Hide prompt and output' button. 3. Add a new option under 'code-block' to control the displaying of the doctest directives at the code-block level. This would override the global setting. I think this option could be added locally, but may need to be done upstream in Sphinx. [1] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#liter... [2] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#docte... [3] http://www.sphinx-doc.org/en/stable/usage/extensions/doctest.html [4] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/directives.html#s... ---------- nosy: +cheryl.sabella _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: I opened an issue on the sphinx-doc repo [1] to check if it would be possible to have an option in doctest blocks to not trim them. We previously had a hack in Doc/tools/extensions/pyspecific.py to patch sphinx to not trim them for the doctest.rst file. But sphinx deprecated the hack we used :( [1]: https://github.com/sphinx-doc/sphinx/issues/6698 ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Change by Julien Palard <julien+python@palard.fr>: ---------- pull_requests: +15647 pull_request: https://github.com/python/cpython/pull/16024 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Gregory P. Smith <greg@krypto.org> added the comment: New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith (Julien Palard) in branch 'master': bpo-36675: Remove obsolete code. (GH-16024) https://github.com/python/cpython/commit/2c910c1e732c9a3ec4c67a7c43d789d6c72... ---------- nosy: +gregory.p.smith _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>: ---------- pull_requests: +15653 pull_request: https://github.com/python/cpython/pull/16030 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment: New changeset 94a684734f669eab02b5c915394b749ccf936449 by Miss Islington (bot) in branch '3.8': bpo-36675: Remove obsolete code. (GH-16024) https://github.com/python/cpython/commit/94a684734f669eab02b5c915394b749ccf9... ---------- nosy: +miss-islington _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Jim DeLaHunt <from+python@jdlh.com> added the comment: We discovered and fixed this same problem back in 2011-2012 with #12947 . That was apparently the source of the monkeypatch that was removed as "obselete code" on 2019-09-12. That old issue commentary has some suggestions about other workarounds. This includes adding explanatory text about the fact that doctest directives are missing from the examples which should be showing them. Maybe some of those workarounds would be effective for us now. ---------- nosy: +JDLH _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Change by Julien Palard <julien+python@palard.fr>: ---------- pull_requests: +22488 pull_request: https://github.com/python/cpython/pull/23620 _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: New changeset c8a10d2fabff492ab352501c316baf5f2fc6510e by Julien Palard in branch 'master': bpo-36675: Doc: Reveal doctest directives (GH-23620) https://github.com/python/cpython/commit/c8a10d2fabff492ab352501c316baf5f2fc... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: Happy Christmas, everybody involved in this issue! I'm happy to announce this issue is resolved since a few days \o/ ---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Anthony Sottile <asottile@umich.edu> added the comment: should the minimum sphinx version be bumped and/or this reverted: https://github.com/python/cpython/blob/f7f0ed59bcc41ed20674d4b2aa443d3b79e72... this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04 The directive used here requires sphinx>=3.2.0 I notice some other attempts have been made to make the docs more compatible with sphinx 1.x in this release as well so there might be conflicting directions here: https://github.com/python/cpython/commit/b63a620014b67a6e63d10783149c41baaf5... ---------- nosy: +Anthony Sottile _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Julien Palard <julien+python@palard.fr> added the comment: Due to https://github.com/python/cpython/pull/24282 this is sadly un-fixed. Either we find another way to fix this, either we wait 3 releases and we re-commit https://github.com/python/cpython/pull/23620. ---------- resolution: fixed -> later stage: resolved -> status: closed -> open _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
Jim DeLaHunt <from+python@jdlh.com> added the comment: My goodness, things get complex sometimes. If we cannot make Sphinx preserve doctest directives and comments, perhaps we should go back to the historical bug discussion to look at workarounds which we considered earlier. For instance, maybe we should modify the text surrounding the examples to explain that doctests directives should appear there, but that our tool chain currently removes them. At the moment, I see doctest directives in the doctest source code at: https://github.com/python/cpython/blob/master/Doc/library/doctest.rst#direct... which do not appear in the corresponding HTML output at: https://docs.python.org/3/library/doctest.html#directives How about rewording the text before each of those examples? Or maybe finding some way to show those examples as literal text which Sphinx won't modify, even if it is not formatted like Python code examples? By the way, the discussion of this same problem back in 2011-2012 is at #12947 . At that time, we applied a "monkey patch" to the Sphinx code. I haven't read the discussion closely enough to figure out if such a patch would help now. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue36675> _______________________________________
participants (12)
-
Anthony Sottile -
Cheryl Sabella
-
Gregory P. Smith
-
Jeroen Demeyer
-
Jim DeLaHunt
-
Julien Palard
-
Karthikeyan Singaravelan
-
miss-islington
-
Roundup Robot
-
Steven D'Aprano
-
Terry J. Reedy
-
Éric Araujo