[Python-Dev] Docutils/reStructuredText is ready to process PEPs

[David Goodger]

Barry A. Warsaw wrote:
> An of course RFCs are also converted to html:
> http://www.faqs.org/rfcs/rfc2822.html

So they are.  Pretty picture at the top, navigation bar at top &
bottom, and a huge <PRE> in-between (with live RFC links at least).
Impressive.  ;-)

> Taken to the extreme, why do we even use a text based format at all?
> We could, of course, get all that by authoring the PEPs directly in
> HTML.

To answer your hypothetical (I assume), it's because raw HTML/XML/SGML
is unreadable to most people.  Plaintext is a common denominator,
useful because it's universally readable.  But texts like RFCs and
PEPs do have some structure; by formalizing that structure we can use
it.  Current PEPs are one up on RFCs, recognizing section titles for
HTML.  ReStructuredText just takes that further.

>     >> I'd like for PEP authors to explicitly choose one or the other,
>     >> preferrably by file extension (e.g. .txt for plain text .rst or
>     >> .rest for reST).
> 
>     DG> I'm not keen on a new file extension (this issue has come up
>     DG> before).  There's so much in place on many platforms that says
>     DG> .txt means text files, and reStructuredText files *are* text
>     DG> files, with just a bit of formal structure sprinkled over.
>     DG> Browsers know what to do with .txt files; they wouldn't know
>     DG> what to do with .rest or .rtxt files.  Near-universal file
>     DG> naming conventions are not the place to innovate IMHO.
> 
> Don't most servers default to text/plain for types they don't know?
> I'm pretty sure Apache does.

I don't know the answer to that.  There's still what the browser does
with it at the client end, and what apps like Windows Explorer and
Mac's File Exchange do with file extensions.  I think those
side-effects make keeping .txt worth it.

> If a file extension isn't acceptable, then I'd still want the
> determination of plaintext vs. reST to be explicit.  The other
> alternative is to add a PEP header to specify.  I'd propose calling
> it Content-Type: and use text/x-rest as the value.

[Already replied to in private email; repeated here to elicit
opinions.]

Good idea.  But the header "Content-Type: text/x-rest" seems to imply
much more than is intended.  PEP 258 proposes a __docformat__ variable
to contain the name of the format being used for docstrings; perhaps a
"Format:" header for PEPs?  For example:

    Format: reStructuredText

Alternatively:

    Format: RST

I prefer "RST" to "rest", which is already used as an acronym for the
"Representational State Transfer" protocol (see Paul Prescod's article
at http://www.xml.com/pub/a/2002/02/20/rest.html).

The existing format could be called "Plaintext" (or "PEP 1.0" ;-).
Without the "Format:" header, "Plaintext" would be the default.

[In his reply to the aforementioned private email,]

Barry pointed out:

    Since the PEP headers are modeled on RFC 2822, I say we stick with
    established standards rather than invent our own.  So
    "Content-Type: text/x-rest" seems natural, and for most related
    standards, if there is no Content-Type: header, text/plain is
    already the documented default.

Looking at the relevant standards (RFC 2616 etc.) I see his point.
Using "Content-type:" may seem like overkill now, but it's flexible
and future-proof (!).  The "charset:" part could also come in handy;
already, there are some PEPs (including PEP 0) which implicitly use
Latin-1.

But "text/x-rst" would be better. :-)

> Fair enough.  Let's do this: send me a diff against v1.39 of
> pep2html.py.

Will do.

> I just downloaded docutils-0.2, but I'm not sure of the
> best way to integrate this in the nondist/peps directory.
> 
> - If we do the normal setup.py install, that's fine for my machine but
>   it means that everyone who will be pushing out peps will have to do
>   the same.
> 
> - If we hack pep2html.py to put ./docutils-0.2 on sys.path, then we
>   can just check this stuff into the peps directory and it should Just
>   Work.  We'd have to update it when new docutils releases are made.

The "docutils" package could be a subdirectory of nondist/peps under
CVS.  When pep2html.py is run, the current working directory is
already on the path so "import docutils" should just work and no
sys.path manipulation would be necessary.  But Docutils is
substantial and evolving.  I don't mind keeping Python's repository in
sync but would others object to the added files and CVS traffic?
Eventually I hope for Docutils to go into the stdlib, but it's not
ready for consideration yet.

I agree with the direct email consensus that "python setup.py install"
is best.

> OTOH, if plaintext PEPs work without having access to the docutils
> package, that would be fine too (another reason perhaps for an
> explicit flag).

Your wish is my command.  If Docutils isn't installed and pep2html.py
is asked to process a reStructuredText PEP, it will report the problem
and move on gracefully (no traceback).

>     DG> Sure.  Once authors see what the new markup gives them, I'm
>     DG> sure there will be some converts.
> 
> Let's find out.

Great.  I'll work on pep2html.py, a README, and a new "Template"
Meta-PEP including a recommended reStructuredText subset.

-- 
David Goodger  <goodger@users.sourceforge.net>  Open-source projects:
  - Python Docutils: http://docutils.sourceforge.net/
    (includes reStructuredText: http://docutils.sf.net/rst.html)
  - The Go Tools Project: http://gotools.sourceforge.net/