[Python-Dev] Escaping docs markup in NEWS entries?

[Wes Turner]

On Tue, May 2, 2017 at 1:58 PM, David Goodger <goodger at python.org> wrote:

> On Mon, May 1, 2017 at 5:03 AM, Wes Turner <wes.turner at gmail.com> wrote:
> > Where would be a good place for test cases for an rst_escape() function?
> > Docutils?
> >
> > https://github.com/westurner/dotfiles/blob/develop/scripts/
> git-changelog.py
>
> I'd say that the test cases for any function should reside in the same
> project/codebase as the function itself. Where do you intend the
> function to reside?
>
> I'm not certain what rst_escape() is supposed to do; what is its
> purpose? The docstrings of rst_escape() as well as the
> git-changelog.py module lack such explanations.
>

def rst_escape(_str: str) -> str:
    """"
    Args:
        _str: string to safely escape for literal inclusion in a
ReStructuredText document
    Returns:
        str: safely-escaped ReStructuredText
    """"


>
> I am also uncertain if this is necessary, based on the remainder of
> the thread. Perhaps it would be better to fix this "make suspicious"
> so that it doesn't produce spurious results? (Note: I am unclear on
> exactly what "make suspicious" is supposed to do, or why.)
>

An rst_escape(text: ) function is necessary in order to prevent
ReStructuredText injection (and e.g. * unintentional partial italicization).

There are other directives for including text in a ReStructuredText
document, e.g.:

- ``.. include::``
- ``.. raw::``

And there are extensions for changelogs and sphinx builds:

- ``.. git_changelog::`` -- https://pypi.org/project/sphinx-git/
- ``.. changelog::``, ``.. change::`` -- https://pypi.org/project/changelog/


But, for a README.rst in a git repository, for example, it's far easier to
diff the document with the data included than it is to have to build from a
source .rst to a dest .rst.rst and then diff the .rst.rst files.

There are CWE URLs for describing the type of vulnerability that a
community-reviewed rst_escape() function would be designed to prevent:

- CWE-116: Improper Encoding or Escaping of Output
  https://cwe.mitre.org/data/definitions/116.html
  - CWE-74: Improper Neutralization of Special Elements in Output Used by a
Downstream Component ('Injection')
    https://cwe.mitre.org/data/definitions/74.html#Relationships

    - CWE-89: SQL Injection
      https://cwe.mitre.org/data/definitions/89.htm

      - 2011 Top25 #1 issue
        https://cwe.mitre.org/top25/#CWE-89

    - [ ] CWE-???: RST Injection


Not a bug in docutils; a PEBKAM implementation "BUG,SEC" issue


>
> I apologize for the azure hue of my answers, to questions that came
> from out of the blue.
> ;-)
>
> David Goodger
> <http://python.net/~goodger>
>
>
> > - rst_escape # YMMV
> > - $ git-changelog.py -r "release/0.3.14" --hdr= "+"`
> >
> > On Monday, May 1, 2017, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >>
> >> On 1 May 2017 at 17:13, Martin Panter <vadmium+py at gmail.com> wrote:
> >> > On 1 May 2017 at 06:37, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> >> Hi folks,
> >> >>
> >> >> I'm trying to write a NEWS entry that explains that the
> >> >> ":func:`bytes`" cross-references have changed to refer to the type
> >> >> descriptions by default (matching other builtin container types), so
> >> >> you now need to use ``:ref:`func-bytes`" to refer to the old target
> in
> >> >> the list of builtin functions (if you really want that for some
> >> >> reason).
> >> >>
> >> >> Unfortunately, my first two attempts both cause warnings in "make
> >> >> suspicious" with the following output:
> >> >
> >> > What is the full output? Usually it includes instructions to add false
> >> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
> >> > do?
> >>
> >> You're right, that would be likely be the way to go if I decided to
> >> keep the escaped markup.
> >>
> >> However...
> >>
> >> >> My fallback plan is to just include the unescaped markup in the NEWS
> >> >> entry (rather than trying to make it readable even in rendered form),
> >> >> but I figured I'd ask for advice here first.
> >> >
> >> > I thought the NEWS file was mainly regarded as plain text, so it would
> >> > be important to avoid ugly RST markup like backslashes if possible.
> >>
> >> ... I think you're right on this point, so it makes more sense to skip
> >> the escaping entirely,
> >> and just use the correct link markup in the NEWS entry.
> >
> > How convenient.
> >>
> >>
> >> Cheers,
> >> Nick.
> >>
> >> --
> >> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170502/f8c79e21/attachment-0001.html>