[Python-Dev] Re: User extendable literal modifiers ?!

[Alex Martelli]

On Friday 27 September 2002 01:03 pm, Gareth McCaughan wrote:
	...
> Better would be user-definable *prefixes*.

Yes -- nice idea.

>     Its return value will replace the $x"..." combination in
>     the token stream, as a literal token.

Why just one token, and why just literal.  Returning an
arbitrary sequence of tokens seems more natural.  This
would allow e.g. Tim Berners-Lee to have basically what
he wants (and asked for in his talk at IPC10) in terms of
extended syntax for graphs, just with some $x in front.

I had a similar idea right after Tim's talk, but could not
articulate it clearly enough in a chat with Guido right
afterwards, and later I didn't follow through with it.  It
seems to me that your proposal is detailed and precise
enough (while my idea was rather vague) and that, by
returning an arbitrary sequence of tokens, it will let
Tim embed whatever funky syntax it requires.

This power is also the downside of the whole idea of
course -- no guarantee that somebody can't use this
mechanism to produce highly obfuscated programs.
But I think that such a somebody could already
obfuscate quite effectively in other ways, and the
risk of abuse shouldn't stop this interesting proposal.

>         ...     return Rational(numerator, denominator)

Hmmm, how would this "return a literal token"?  It returns
an instance of Rational -- how does the parser treat this
instance as a literal token?

I thought this use would have to return the sequence of
tokens for identifier 'Rational', open parenthesis, literal
(value of) numerator, comma, literal (value of) denominator,
closed parenthesis -- which in turn is why I thought of an
arbitrary sequence of tokens.  If a single instance of any
arbitrary class may be returned and get treated as a
literal token by the parser, then that's much better (maybe
I don't know Python's parser well enough, but I don't
clearly see how that would be done).

>   - Is this insane?

Hope not, since I like it.

>   - Is "$" the best character?

Among the few available ones, I think I slightly prefer "@"
for this use, but there's little to choose IMHO.


Alex