[issue36675] Doctest directives and comments not visible or missing from code samples
New submission from Steven D'Aprano steve+python@pearwood.info:
(Apologies if this is the wrong place for reporting website bugs.)
The website is not rendering doctest directives or comments, either that or the comments have been stripped from the examples.
On the doctest page itself, all the comments are missing:
https://docs.python.org/3/library/doctest.html#directives
The first example says:
>>> print(list(range(20))) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19]
but without the directive, the test would fail. Screen shot attached.
Doctest directives are also missing from here:
https://docs.python.org/3/library/ctypes.html
My browser: Firefox 45.1.1
Also checked with text browser "lynx".
---------- assignee: docs@python components: Documentation files: missing_directives.png messages: 340570 nosy: docs@python, steven.daprano priority: normal severity: normal status: open title: Doctest directives and comments not visible or missing from code samples type: behavior Added file: https://bugs.python.org/file48277/missing_directives.png
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Change by Karthikeyan Singaravelan tir.karthi@gmail.com:
---------- nosy: +mdk
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Terry J. Reedy tjreedy@udel.edu added the comment:
I verified that the line in Doc/library/doctest.rst has the comment. " For example, this test passes::
print(list(range(20))) # doctest: +NORMALIZE_WHITESPACE
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] " However, the comment is omitted from the .html built on Windows by Sphinx 1.8.1. " <div class="highlight-python3 notranslate"><div class="highlight"><pre><span></span><span class="gp">>>> </span><span class="nb">print</span><span class="p">(</span><span class="nb">list</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">20</span><span class="p">)))</span> ***html for comment should be here*** <span class="go">[0, 1, 2, 3, 4, 5, 6, 7, 8, 9,</span> <span class="go">10, 11, 12, 13, 14, 15, 16, 17, 18, 19]</span> </pre></div> </div> " To me, this is a bug with building the .html for doctest.rst. Comments are preserved elsewhere. For instance, the code example for https://docs.python.org/3/library/functions.html#dir. The .rst file has " The resulting list is sorted alphabetically. For example:
>>> import struct >>> dir() # show the names in the module namespace # doctest: +SKIP " Both use 3-space colon + indents to mean 'example block'. The only difference is '::' versus ':'. https://devguide.python.org/documenting/#source-code says :: is required. idle.rst also has a code example with comments displayed (I just submitted a PR to not suppress color highlighting.)
I leave it to the doc experts to discover why the comments are not included in doctest.html.
---------- nosy: +eric.araujo, ezio.melotti, terry.reedy, willingc stage: -> needs patch title: Doctest directives and comments not visible or missing from code samples -> Doctest directives and comments missing from code samples versions: +Python 2.7, Python 3.7, Python 3.8
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Change by Roundup Robot devnull@psf.upfronthosting.co.za:
---------- keywords: +patch pull_requests: +13038 stage: needs patch -> patch review
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Jeroen Demeyer J.Demeyer@UGent.be added the comment:
Isn't it a *feature* that those doctest directives are not shown? Those directives are meant for the doctest module only, not for the reader of the rendered documentation.
---------- nosy: +jdemeyer
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Éric Araujo merwok@netwok.org added the comment:
OP is about the documentation page for doctest itself!
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Terry J. Reedy tjreedy@udel.edu added the comment:
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives. The patch only exposes them for doctest.html and not for ctypes or anywhere else.
They really should not be in the dir example code that I linked to. https://docs.python.org/3/library/functions.html#dir The problem there are the double comments with both a real comment and a directive.
https://devguide.python.org/documenting/#source-code does not say anything about '::' causing suppression of comments and ':' leaving them. It is misleading in implying the '::' is required for a code block.
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Steven D'Aprano steve+python@pearwood.info added the comment:
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives. The patch only exposes them for doctest.html and not for ctypes or anywhere else.
Thanks for the patch, and the extra information, but I disagree with the decision to suppress the directives.
The reason I found this problem in the first case was that I started with the ctypes documentation, where it says:
"Since some code samples behave differently under Linux, Windows, or Mac OS X, they contain doctest directives in comments."
and I was very keen to see those directives so I could learn the right way to deal with platform-dependent doctests. I was very confused that they weren't visible.
https://docs.python.org/3/library/ctypes.html
I think that doctest directives are as much a part of documenting correct usage as any other part of the example code, and they are (semi-)human readable and (almost) self-documenting.
Consider this example from ctypes:
>>> c_wchar_p("Hello, World") c_wchar_p(140018365411392)
In the absence of a directive, but knowing that it may have been surpressed, I don't know how to interpret that. Is the output some arbitrarily chosen value that doctest ought to skip? Or is that the actual output that c_wchar_p("Hello, World") will return every single time without fail?
If I was a ctypes expert, it might be blindingly obvious to me, but I'm not, so I'm left in the dark. I don't know whether I should expect that precise output each and every time, or something platform and implementation specific.
If the directive #doctest:+SKIP was visible, I would know that it was an arbitrarily chosen example.
My preference would be:
- keep the doctest directives visible, everywhere;
- make them a clickable link to the appropriate section in the doctest documentation;
- and, if possible, on mouse-over, they should display a tooltip with a message like
"The output of this example is arbitrary."
Or similar wording.
They really should not be in the dir example code that I linked to. https://docs.python.org/3/library/functions.html#dir
On the contrary: I think that the presence of the +SKIP directive helps demonstrate that the output shown is a made-up example, not normative.
(Of course it helps that I know doctest, but even if I didn't, the tooltip message would help.)
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Jeroen Demeyer J.Demeyer@UGent.be added the comment:
Doctest directives in code examples should be suppressed everywhere *except* in the doctest.html examples showing how to use directives.
Thanks for clarifying. I missed that.
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Cheryl Sabella cheryl.sabella@gmail.com added the comment:
;tldr; There is a global configuration flag to show all the doctest directives for all the docs that are built. The default is to suppress the doctest directives. If a global setting isn't desired, then each doctest example will have to be changed individually.
I've looked at the Sphinx documentation for this and there are a few points to reference.
* `::` defines a Literal block in reST. [1] A literal block doesn't do anything special except from the last paragraph is this section of the docs: "Code highlighting can be enabled for these literal blocks on a document-wide basis using the `highlight` directive and on a project-wide basis using the `highlight_language` configuration option. The code-block directive can be used to set highlighting on a block-by-block basis. These directives are discussed later." CPython has the `highlight-language` set to python3.
* A doctest block does not need to be in a Literal Block. [2] They are identified automatically. [3] According to the doc, doctest directives are suppressed from presentation:
Also, you can give inline doctest options, like in doctest:
>>> datetime.date.now() # doctest: +SKIP datetime.date(2008, 1, 1)
They will be respected when the test is run, but stripped from presentation output.
* Any section (doesn't need the single colon (`:`) can have the `code-block` directive on it. [4] The `code-block` directive can define the language for highlighting.
Note that in the Sphinx doctest [4] documentation, there is a config option `trim_doctest_flags`, which is True by default. Setting this to False will show all the doctest directives for all the doctests in the documentation.
The nice thing about the doc build recognizing a section as a doctest is that it adds the toggle in the upper right of the block to 'Hide the toggle and output' so that the example can be copy and pasted more easily. Changing it into something other than a doctest (like using a `code-block` directive, takes away this option. Using `trim_doctest_flags=False` retains the hide button.
Options: 1. Change global setting to false to show all doctest directives. 2. Change only the blocks where the doctest directives need to be shown, but this makes them lose the 'Hide prompt and output' button. 3. Add a new option under 'code-block' to control the displaying of the doctest directives at the code-block level. This would override the global setting. I think this option could be added locally, but may need to be done upstream in Sphinx.
[1] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#liter... [2] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/basics.html#docte... [3] http://www.sphinx-doc.org/en/stable/usage/extensions/doctest.html [4] http://www.sphinx-doc.org/en/stable/usage/restructuredtext/directives.html#s...
---------- nosy: +cheryl.sabella
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Julien Palard julien+python@palard.fr added the comment:
I opened an issue on the sphinx-doc repo [1] to check if it would be possible to have an option in doctest blocks to not trim them.
We previously had a hack in Doc/tools/extensions/pyspecific.py to patch sphinx to not trim them for the doctest.rst file. But sphinx deprecated the hack we used :(
[1]: https://github.com/sphinx-doc/sphinx/issues/6698
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Change by Julien Palard julien+python@palard.fr:
---------- pull_requests: +15647 pull_request: https://github.com/python/cpython/pull/16024
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Gregory P. Smith greg@krypto.org added the comment:
New changeset 2c910c1e732c9a3ec4c67a7c43d789d6c729304a by Gregory P. Smith (Julien Palard) in branch 'master': bpo-36675: Remove obsolete code. (GH-16024) https://github.com/python/cpython/commit/2c910c1e732c9a3ec4c67a7c43d789d6c72...
---------- nosy: +gregory.p.smith
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Change by miss-islington mariatta.wijaya+miss-islington@gmail.com:
---------- pull_requests: +15653 pull_request: https://github.com/python/cpython/pull/16030
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
miss-islington mariatta.wijaya+miss-islington@gmail.com added the comment:
New changeset 94a684734f669eab02b5c915394b749ccf936449 by Miss Islington (bot) in branch '3.8': bpo-36675: Remove obsolete code. (GH-16024) https://github.com/python/cpython/commit/94a684734f669eab02b5c915394b749ccf9...
---------- nosy: +miss-islington
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Jim DeLaHunt from+python@jdlh.com added the comment:
We discovered and fixed this same problem back in 2011-2012 with #12947 .
That was apparently the source of the monkeypatch that was removed as "obselete code" on 2019-09-12. That old issue commentary has some suggestions about other workarounds. This includes adding explanatory text about the fact that doctest directives are missing from the examples which should be showing them. Maybe some of those workarounds would be effective for us now.
---------- nosy: +JDLH
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Change by Julien Palard julien+python@palard.fr:
---------- pull_requests: +22488 pull_request: https://github.com/python/cpython/pull/23620
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Julien Palard julien+python@palard.fr added the comment:
New changeset c8a10d2fabff492ab352501c316baf5f2fc6510e by Julien Palard in branch 'master': bpo-36675: Doc: Reveal doctest directives (GH-23620) https://github.com/python/cpython/commit/c8a10d2fabff492ab352501c316baf5f2fc...
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Julien Palard julien+python@palard.fr added the comment:
Happy Christmas, everybody involved in this issue! I'm happy to announce this issue is resolved since a few days \o/
---------- resolution: -> fixed stage: patch review -> resolved status: open -> closed
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Anthony Sottile asottile@umich.edu added the comment:
should the minimum sphinx version be bumped and/or this reverted: https://github.com/python/cpython/blob/f7f0ed59bcc41ed20674d4b2aa443d3b79e72...
this change breaks, for example, sphinx 1.8.5 while building for ubuntu 20.04
The directive used here requires sphinx>=3.2.0
I notice some other attempts have been made to make the docs more compatible with sphinx 1.x in this release as well so there might be conflicting directions here: https://github.com/python/cpython/commit/b63a620014b67a6e63d10783149c41baaf5...
---------- nosy: +Anthony Sottile
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Julien Palard julien+python@palard.fr added the comment:
Due to https://github.com/python/cpython/pull/24282 this is sadly un-fixed.
Either we find another way to fix this, either we wait 3 releases and we re-commit https://github.com/python/cpython/pull/23620.
---------- resolution: fixed -> later stage: resolved -> status: closed -> open
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
Jim DeLaHunt from+python@jdlh.com added the comment:
My goodness, things get complex sometimes.
If we cannot make Sphinx preserve doctest directives and comments, perhaps we should go back to the historical bug discussion to look at workarounds which we considered earlier. For instance, maybe we should modify the text surrounding the examples to explain that doctests directives should appear there, but that our tool chain currently removes them.
At the moment, I see doctest directives in the doctest source code at: https://github.com/python/cpython/blob/master/Doc/library/doctest.rst#direct... which do not appear in the corresponding HTML output at: https://docs.python.org/3/library/doctest.html#directives
How about rewording the text before each of those examples? Or maybe finding some way to show those examples as literal text which Sphinx won't modify, even if it is not formatted like Python code examples?
By the way, the discussion of this same problem back in 2011-2012 is at #12947 . At that time, we applied a "monkey patch" to the Sphinx code. I haven't read the discussion closely enough to figure out if such a patch would help now.
----------
_______________________________________ Python tracker report@bugs.python.org https://bugs.python.org/issue36675 _______________________________________
participants (12)
-
Anthony Sottile -
Cheryl Sabella
-
Gregory P. Smith
-
Jeroen Demeyer
-
Jim DeLaHunt
-
Julien Palard
-
Karthikeyan Singaravelan
-
miss-islington
-
Roundup Robot
-
Steven D'Aprano
-
Terry J. Reedy
-
Éric Araujo