[Python-ideas] User-defined literals

[Petr Viktorin]

On Fri, Jun 5, 2015 at 2:09 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 4 June 2015 at 23:31, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> The fundamental difference between this proposal and mine is (I think)
>>> that you're assuming an arbitrary Python expression in there (which is
>>> parsed), whereas I'm proposing an *unparsed* string.
>>
>> No, when you supplied a custom parser, the parser would have access to the
>> raw string (as well as the name -> cell mapping for the current scope).
>>
>> The "quoted AST parser" would just be the default one.
>
> Ah, I see now what you meant. Apologies, I'd not fully understood what
> you were proposing. In which case yes, your proposal is strictly more
> powerful than mine.
>
> You still have the same problem as me, that what's inside !xxx(...)
> cannot contain a ")" character. (Or maybe can't contain an unmatched
> ")", or an unescaped ")", depending on what restrictions you feel like
> putting on the form of the unparsed expression...) But I think that's
> fundamental to any form of syntax embedding, so it's not exactly a
> showstopper.

Parsing consumes tokens. The tokenizer already tracks parentheses (for
ignoring indentation between them), so umatched parens would throw off
the tokenizer itself.
It'd be reasonable to require !macros to only contain valid Python
tokens, and have matched parentheses tokens (i.e. ignoring parens in
comments/string literals.)