[Doc-SIG] extensibility (was re: reStructuredText inline markup)

[David Goodger]

extensibility (was re: reStructuredText inline markup)

[Alan]
> Extensibility beyond the current "roles" mechanism is also
> necessary. For example, one of my intended applications for rST
> currently has a frequently-used tag to refer to another user of the
> system::
> 
>     I had lunch with <lj user="tikva" text="Rachel"> today.
> 
> It's a nice semantically meaningful output-format-neutral element.

It's also markup-heavy and very application specific. If you want that
level of markup, you'll have to live with markup that looks like
markup.

> The extensibility problem is relatively easy to address if you allow
> attributes or arguments in roles::
> 
>     I had lunch with :lj user=tikva:`Rachel` today.

Instead, think of the role as the tag name; you can parse the text
between the backquotes however you like.

> But to me, that syntax looks confusing. I was never too fond of the
> `` :rolename:`text` `` syntax in the first place, and adding spaces
> makes it more confusing.

It's a last resort, for application-specific constructs. It's useful
in moderation, but I hope that the ``:role:`text``` syntax is never
abused.

> I'd also like to require that it be postfix, both for simplicity,
> and to follow the general principle that the text should be primary
> and the way it's marked up should be an aside. ::
> 
>     I had lunch with `Rachel`{lj user=tikva} today.

As Ueli pointed out, postfix is OK too.

[Ueli]
> And it would be entirely up to you to use some mail-header inspired
> quoting mechanism, i.e.::
> 
>     I had lunch with `Rachel <tikva>`:lj: today.

[Alan]
> That works well in this case with only one argument that isn't too
> distracting when placed inline and has an evocative shorthand.

And if this is the main or only use of interpreted text in your
application, you could drop the role altogether::

    I had lunch with `Rachel <tikva>` today.

Maybe you could do a text to username lookup in your app::

    I had lunch with `Rachel` today.

[Alan]
> But imagine::
> 
>     The `text="biohazard" src="biohazard.png" height=20
>     width=20`:img: symbol must be used on containers used to dispose
>     of medical waste.
> 
> That really needs to get placed yanked out of the flow of text.

What may be needed is the equivalent of SGML/XML's named entities.
Something like this (syntax arbitrary and subject to debate)::

    The #biohazard# symbol must be used on containers used to dispose
    of medical waste.

    .. substitution:: biohazard
       text="biohazard" source="biohazard.png" height=20 width=20

(Although the "text=" attribute may be redundant and unnecessary.)
``#biohazard#`` would be replaced by whatever the ``substitution::
biohazard`` directive generates.

Instead of ``#name#``, the syntax could be a variant of interpreted
text too. Something like::

    `name`@ or `name`# or `name`&

[Alan]
> If we think of these as inline directives, we can steal a play from
> the hyperlink book, and drag the role out-of-line::
> 
>     I had lunch with `Rachel`{_} today.
> 
>     .. _Rachel: {lj user=tikva icon="badger.png"}
> 
> There's even the possibility of::
> 
>     I had lunch with `Rachel`{__} today.
> 
>     __ {lj user=tikva icon="badger.png"}

You do like curly braces, don't you? ;-)

What is the end result of this markup? Is it going to end up as a
hyperlink? Please show us how you would mark it up in HTML.

[Ueli]
> How about::
> 
>     I had lunch with `Rachel`_ today.
> 
>     .. _Rachel:
>     
>        .. lj:: user=tikva icon="badger.png"
> 
> I'm not sure about the idented directive -- I could find no explicit
> exclusion, so I assume that it is correct syntax.

It's not included in the three things allowed in a link block: nothing
or empty (internal hyperlink target), URI (external target), reference
(indirect target). I suppose a hyperlink target's link block could be
generalized to allow a directive also ("custom hyperlink target"?), but
I'm not sure if it's warranted for the general case.

And even if directives were allowed inside hyperlink targets' link
blocks, they would still need to produce a hyperlink target.
Non-hyperlink cases wouldn't benefit.

(Note that the way the syntax works now, the link block would have to
immediately follow the target marker, with no blank line in-between,
unlike Ueli's example.)

[Ueli]
> But is it necessary, or would this do as well (similar to multiple
> targets for a hyperlink)::
> 
>     I had lunch with `Rachel`_ today.
> 
>     .. _Rachel:
>     .. lj:: user=tikva icon="badger.png"

That depends on what the "lj" construct actually does. Looking at the
above, I'd expect "Rachel" to be a reference to a picture or object
where the ``.. _Rachel:`` target points.

[Alan]
> Now we're getting into using that target syntax for not only hrefs
> and anchors, but also macro inclusion. rST doesn't currently have
> anything which includes referenced text at the current position in
> the document. Not necessarily a bad idea, and macro-including
> directives would solve the inline markup issue, but it's getting
> into new territory.

It may be the right territory though. Generalizing further, the
"substitution" directive could contain text, or a directive which
resolves to text or an image or some other inline-compatible object.

> it looks odd when you make the interpreted text into a hyperlink::
> 
>     The `biohazard`:_:_ symbol ...
> 
> My brain tokenizes that as ``:_ :_``, while it does the right thing
> with ``{_}_``, turning it into ``{_} _``.

That's *horrible*! This is supposed to be a readable markup. We must
exercise restraint!

[Ueli]
> Doesn't the current spec_ cover your wishes better than you seem to
> think?  The spec says clearly that the role may be prefix or postfix,
> and that the interpretation is domain-dependent.
> 
> .. _spec: *reStructuredText Markup Specification*, CVS version 1.22
>            of 2001/10/27

[Alan, referring to Ueli's ``.. _spec:`` construct above]
> Is that valid?  The reference processor says::
> 
>     Warning: [level 1] Hyperlink target at line 5 contains
>     whitespace. Perhaps a footnote was intended?

It probably should be a footnote. Notice how helpful the parser is?

-- 
David Goodger    goodger@users.sourceforge.net    Open-source projects:
 - Python Docstring Processing System: http://docstring.sourceforge.net
 - reStructuredText: http://structuredtext.sourceforge.net
 - The Go Tools Project: http://gotools.sourceforge.net