[issue28710] Sphinx incompatible markup in configparser.ConfigParser.
New submission from Patrick Lehmann: Why does e.g. configparser.ConfigParser contain doc strings with Sphinx incompatible markup? The markup starts with back-tick, but ends with a single quote. Example: https://github.com/python/cpython/blob/master/Lib/configparser.py?ts=2#L26 Sphinx writes these messages: D:\git\PoC\py\lib\ExtendedConfigParser\__init__.py:docstring of lib.ExtendedConfigParser.ExtendedConfigParser.read_file:3: WARNING: Inline interpreted text or phrase reference start-str ing without end-string. Note: ExtendedConfigParser is class derived from configparser.ConfigParser. Inherited methods get documented too. ------------------------ Btw. I have some improvements for this class, how can I contribute them? Who is the maintainer for this class? Please contact me: Patrick.Lehmann@tu-dresden.de The improved version is available at GitHub: https://github.com/Paebbels/ExtendedConfigParser?ts=2 ---------- assignee: docs@python components: Documentation messages: 280907 nosy: Patrick Lehmann, docs@python priority: normal severity: normal status: open title: Sphinx incompatible markup in configparser.ConfigParser. versions: Python 3.4, Python 3.5 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Changes by Serhiy Storchaka <storchaka+cpython@gmail.com>: ---------- keywords: +easy stage: -> needs patch versions: +Python 2.7, Python 3.6, Python 3.7 -Python 3.4 _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann added the comment: How can I supply a fix? I have a branch with lots of fixes. https://github.com/Paebbels/cpython/tree/paebbels/issue-28710?ts=2 Why don't you accept pull requests via GitHub? Kind regards Patrick ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
R. David Murray added the comment: We will accept github pull requests in the future (the transition is underway). For now, you can create a diff file (using hg diff by preference, but git diff will work) and uploaded it to the issue. For this issue, please only upload the docstring changes. For other enhancement suggestions, open separate issues. (This would be true even if we were accepting pull requests). The existing docstring markup is probably a remnant of the days when the documentation was written in LaTeX. Lukasz Langa is the current maintainer of this module. ---------- nosy: +lukasz.langa, r.david.murray _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann added the comment: Here is the patch file created with: PS> git diff > docstring_markup.patch This patchfile effects all files with this markup in the CPython repository. Kind regards Patrick ---------- keywords: +patch Added file: http://bugs.python.org/file45515/docstring_markup.patch _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy added the comment: As far as I looked, the patch changes `xyz' in docstrings and quotes to ``xyz``. A rst expert should verify that this is correct. In printed strings, `zyz' is changed to 'xyz', which I consider to be correct. Before applying this, I would want to review in Rietveld, with side by side diff and changes color marked. However, Rietveld does not like the patch and there is no 'review' button. I thought it might be the git format, but I found another git patch that did have a review button. When I downloaded and tried to apply (to default), hg says that there is no diff. I don't see the problem, but we cannot currently use a patch that does not apply in hg. Patrick, in order to use patches, we require Contributor Agreements. https://www.python.org/psf/contrib/ There is an electronic form that makes submission easy. https://www.python.org/psf/contrib/contrib-form/ ---------- nosy: +terry.reedy stage: needs patch -> patch review _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
R. David Murray added the comment: I think that we do not generally use ReST markup in our docstrings. So replacing `x' with 'x' would be more correct, I think. In many cases the quotes could just be omitted entirely. The patch command says: patch unexpectedly ends in middle of line patch: **** Only garbage was found in the patch input. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann added the comment: Hello, I used this regexp on all files: -------------------------------------- match pattern: `([A-Za-z0-9_]+)' replace pattern ``\1`` -------------------------------------- I assumed that only identifiers where quoted in such way. I think my editor found around 139 matches in the whole CPython repository. I found some of these markup in non docstring strings, which I reverted as far as I found them by manually reviewing all changed files. For a colored diff, see my Git branch: https://github.com/Paebbels/cpython/commit/6d3f348f71b5b0ae9fbfcb8fdbba72dc5... There is also a PR-, commit-, and line-based comment feature box GitHub. How you solve it is up to you, but I would like to get rid of hundreds of warnings in my Sphinx runs, when modules are inherting code (and docstrings) from Python. Kind regards Patrick ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann added the comment: .... I signed the CLA. ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann added the comment: I also found some docstrings using double back-tick plus double single quotes. For example: ``x.y = v'' in builtins.py in function setattr(...). ---------- _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Changes by Serhiy Storchaka <storchaka+cpython@gmail.com>: ---------- nosy: +georg.brandl _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Łukasz Langa added the comment: Marking as a stdlib-wide issue since it's not configparser-specific. FYI, as you probably noticed, the `term' syntax predates Sphinx and is used in lots of places. While I think it would be nice to fix it, it's a big diff. ---------- title: Sphinx incompatible markup in configparser.ConfigParser. -> Sphinx incompatible markup in the standard library _______________________________________ Python tracker <report@bugs.python.org> <http://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann <Patrick.Lehmann@tu-dresden.de> added the comment: Any progress on that issue? 1.5 years passed by and it should be possible to fix the Python documentation in that time, right? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
R. David Murray <rdmurray@bitdance.com> added the comment: Thanks for coming back to this. We're accepting PRs via github now, so the next step now would be to make it into a PR. Sometimes things just get forgotten and you have to nudge them to get them moving (see the devguide for guidelines about when it is appropriate to nudge an issue). ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: I agree with David about replacing `x' with 'x' instead of ``x``, so please make this change. Do you know what ``x.y = v'' is supposed to mean? David, do you want one PR with 139+ changes, or should we start with fewer? Patrick, are these usages scattered over numerous files or concentrated in a few? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann <Patrick.Lehmann@tu-dresden.de> added the comment: Against what branch should I create the PR? I was a huge number of changes. I think I'll create multiple PRs to ease the review. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: master, and maybe backport from there, unless the PR never touches master. I wonder if we should bring this up on pydev. Without discussion and agreement, someone might object just because there are so many changes. Or someone might say that they want the 'illegal' markup kept. I see that the one docstring you linked to, https://github.com/python/cpython/blob/master/Lib/configparser.py?ts=2#L26 had 9 occurences, 1 for each parameter. I never see any warnings. It must be that our build system somewhere discards them. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment: This should probably be discussed on python-dev. In particular, changing from single backticks to double backticks is something that makes Sphinx happy but will uglify the docstrings for all other ways of viewing the docs. Also, note that most of our docs aren't autogenerated from the docstrings so we've never had any particular need to impose Sphinx markup requirements for docstrings. ---------- nosy: +rhettinger _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: In my last comment above, I forgot that this issue is about *docstrings*. We do not officially process docstrings with Sphinx, so there are no warnings to be suppressed. PEP 8, which covers style for the stdlib, refers to https://www.python.org/dev/peps/pep-0257/ Neither says anything about markup and last I remember, there should not be any for the stdlib. Pep-0257 gives this example: def complex(real=0.0, imag=0.0): """Form a complex number. Keyword arguments: real -- the real part (default 0.0) imag -- the imaginary part (default 0.0) """ Here, parameter names are indicated by the formatting, not by markup. If `' is used in such lists, it should just be deleted. I believe 'name' is sometimes used in running text. The help() function prints a docstring as is. ---------- nosy: -rhettinger _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Change by Raymond Hettinger <raymond.hettinger@gmail.com>: ---------- title: Sphinx incompatible markup in the standard library -> Sphinx incompatible markup in the standard library docstrings _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
R. David Murray <rdmurray@bitdance.com> added the comment: Right, my opinion is that we shouldn't be putting markup in docstrings. They are (from our point of view) pure text. I don't know if discussion on python-dev is warranted, it seems like a fairly uncontroversial change, since it brings the docstrings in question into compliance with our general practice in the majority of the stdlib. Unless my impression about that is wrong :) I don't have an opinion on multiple versus single PR for this. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
R. David Murray <rdmurray@bitdance.com> added the comment: By the way, in case anyone is curious, I'm pretty sure that markup is left over from the days when tex/latex was what docs were *written* in. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: What is the problem? Docstrings are not written in the reStructuredText format in general. If Sphinx complains about docstrings in the stlib, don't run Sphinx for the stdlib files or report a Sphinx bug. Even if remove all `x', docstrings still will not be valid reStructuredText. For example the last three lines will be joined in a single paragraph for the docstring in msg319848. `name' does not have special meaning in (La)TeX. AFAIK `name' is common writing of quotes in English texts. It predates reStructuredText and was widely used in Usenet and mailing lists (as two spaces after sentence-ending punctuation, double hyphen for a dash, _n_a_m_e_ to simulate underscoring, etc). It is just a part of old computer-typing culture (or may be even pre-computer). ---------- nosy: +serhiy.storchaka _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
R. David Murray <rdmurray@bitdance.com> added the comment: No, it is (somewhat) unique to tex. If you write `word' tex would turn that into a pair of opposing quotes in the typeset document, as opposed to 'word' (the convention in regular text/emails/posts/etc) where you'd just see ascii quotes. tex would render 'word' as a closing quote both before and after word, which looks weird in typeset text. There's no bug here; as you say we aren't interested in making the docstrings parseable as restructured text (at least, I'm not). For me, this is about getting rid of the now-odd-looking tex leftovers and making the ascii styling consistent with the bulk of our docstrings. It's not a big deal, though. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment: Oh, you are right. I didn't write much English TeX, and used other types of quotes. I'm sure that I seen `such quotation' in non-TeX files, but maybe my memory fools me. In any case using this writing here is likely an artifact of copying from a TeX documentation. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment:
AFAIK `name' is common writing of quotes in English texts
I don't remember ever seeing it before. It looks like a typo to me, and I am sure it will to most readers. I think it should be corrected as if it were a typo. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Patrick Lehmann <Patrick.Lehmann@tu-dresden.de> added the comment: Having single quotes in docstrings is also ok for Sphinx documentation. Btw. ReStructured text (docutils) was invented to document Python sources, why is it not used by Python? ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: RestructuredText, DocUtils, and Sphinx were developed independently, by people other than the pydev/cpython group. (The proposal to include DocUtils in the stdlib was rejected.) We converted to .rst for the Python documentation sources files about a decade ago. Sphinx turns them into the .html files you can see online. RestructuredText markup can now also be used in PEP, news/changelog, and What's New sources. These are things that most people only view online or in other processed forms, not in source form. On the other hand, stdlib docstrings are mostly viewed unprocessed in .py sources or as output from help() in text consoles or text widgets. So markup in stdlib docstrings would impact everyone, not just core developers. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment: Marking this as closed. Though well-intentioned, the suggestion is predicated on a misperception of the role of Sphinx for CPython and existing PEP recommendations on docstring practices. ---------- nosy: +rhettinger resolution: -> not a bug stage: patch review -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
R. David Murray <rdmurray@bitdance.com> added the comment: I would still like to see the legacy tex markup removed from the docstrings, so I think closing this issue is not appropriate, but it's not a big enough deal that I'll push for it if Raymond wants to keep it closed. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
Terry J. Reedy <tjreedy@udel.edu> added the comment: I agree with David. I would like the ugly markup changed independent of how Sphinx treats it. I was thinking of changing the title to "Change obsolete tex markup in stdlib docstrings". The intent of `word' was for people to see balanced quotes, which in ascii text means 'word'. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue28710> _______________________________________
participants (6)
-
Patrick Lehmann -
R. David Murray
-
Raymond Hettinger
-
Serhiy Storchaka
-
Terry J. Reedy
-
Łukasz Langa