[Doc-SIG] Proposal for indented sections in reStructuredText

[David Goodger]

The vehement feedback from Wolfgang Lipp and Edward Welbourne, as well as
the notes of regret from Ken Manheimer and Doug Hellmann, caused me to
reflect on the real reasons why I reject how StructuredText uses indentation
for sections. I then came up with a form of indented sections that I can
live with, and I hope everyone else can too. The rejection rationale and
proposed solution follow. Please let me know what you think.

StructuredText states: "A single-line paragraph whose immediately succeeding
paragraphs are lower level [indented] is treated as a header." Here's an
example::

    A text paragraph.

    A Section Title

        A text paragraph.

There is very little to distinguish the title from the text paragraphs.
Capitalization, and the lack of final punctuation, give us very subtle
clues; neither is a requirement though. It could just as easily be::

    A section title that looks like a text paragraph.

        A text paragraph.

As can be seen from the examples above, StructuredText's headers are
ambiguous and potentially surprising.

PEPs get around this by requiring sections (and therefore titles and
indentation) for all text. In other words, no paragraphs *can* start at
column 1, so there's no ambiguity about the titles. Unfortunately, this also
limits PEPs to one level of recognized sections. I write more
hierarchically; my PEP 258 uses two levels of subsections.

For the vast majority of docstrings and standalone documents which will
potentially use reStructuredText, only one level of section structure is
necessary. To force the author to use indentation for these documents is
unnecessary and irritating.

Having said all that, and in light of their popularity, I've reconsidered
the value of indented sections. I propose that reStructuredText also allow
entire sections (titles *and* section bodies) to be indented relative to
their enclosing block. Title adornment (an underline and possibly an
overline) will still be required, as well as overall consistency. Once a
section is indented, all subsections must also be indented. Title adornment
style usage must remain consistent.

For example, here are three equivalent variations of a standalone document
with a document title, three sections, and one subsection. Optional blank
lines are omitted for brevity.

1. Unindented reStructuredText::

       ================
        Document Title
       ================

       It is my business to know things. That is my trade.

       Introduction
       ============
       All those moments, lost in time, like tears in rain.

       Main Body
       =========
       Remember: no matter where you go, there you are.

       SubSection
       ----------
       I don't remember yesterday. Today it rained.

       Conclusion
       ==========
       Nobody expects the Spanish Inquisition!

2. Indented subsection only::

       Document Title
       ==============

       It is my business to know things. That is my trade.

       Introduction
       ------------
       All those moments, lost in time, like tears in rain.

       Main Body
       ---------
       Remember: no matter where you go, there you are.

           SubSection
           ----------
           I don't remember yesterday. Today it rained.

       Conclusion
       ----------
       Nobody expects the Spanish Inquisition!

3. Fully indented, sections and subsection, with only one title
   adornment style::

       Document Title
       --------------

       It is my business to know things. That is my trade.

           Introduction
           ------------
           All those moments, lost in time, like tears in rain.

           Main Body
           ---------
           Remember: no matter where you go, there you are.

               SubSection
               ----------
               I don't remember yesterday. Today it rained.

           Conclusion
           ----------
           Nobody expects the Spanish Inquisition!

I examined and rejected the following variation, which is StructuredText
with underlined titles::

       Document Title
       --------------

           It is my business to know things. That is my trade.

           Introduction
           ------------
               All those moments, lost in time, like tears in rain.

           Main Body
           ---------
               Remember: no matter where you go, there you are.

               SubSection
               ----------
                   I don't remember yesterday. Today it rained.

           Conclusion
           ----------
               Nobody expects the Spanish Inquisition!

I rejected the variation above for two reasons:

1. It has section bodies indented relative to the titles, as if the
   bodies 'belong' to the titles and the titles are superior. This
   is contrary to the near-universal model which has titles and
   section bodies belonging to the section itself: titles and bodies
   are at an equal level.

2. Except for the document title, every line of a standalone document
   is indented. This is unacceptably wasteful and irritating.

On reStructuredText's web site (http://structuredtext.sf.net/), I have
modified my arguments in `Problems With StructuredText`_, removing the
'awkward' argument (both indentation and adornment are awkward) and adding
'unnecessary' and 'ambiguous' arguments. I have also modified the
`reStructuredText Markup Specification`_ document.

Please comment; your feedback is welcome.

.. _Problems With StructuredText:
   http://structuredtext.sf.net/spec/problems.txt
.. _ reStructuredText Markup Specification:
   http://structuredtext.sf.net/spec/reStructuredText.txt

-- 
David Goodger    dgoodger@bigfoot.com    Open-source projects:
 - Python Docstring Processing System: http://docstring.sf.net
 - reStructuredText: http://structuredtext.sf.net
 - The Go Tools Project: http://gotools.sf.net