RFC PEP candidate: q'<delim>'quoted<delim> ?

[Neil Hodgson]

Jason Orendorff:
> Neil Hodgson wrote:
> >    In my experience writing a lexer for Perl (for use in syntax styling
> > editors), 'here documents' and arbitrary quote sequences add much
> > complexity to a lexer. [...]
>
> (skeptical look)  Are you really claiming to have written a lexer
> for Perl?  Is that even possible?

   No. I started writing a lexer for Perl, but it is incomplete and is
likely to remain incomplete. The subset of Perl handled is sufficient for
syntax styling but is not sufficient for a language interpreter. The code is
available as part of Scintilla and is used in various IDEs. At one stage it
was used in Komodo, but ActiveState may have changed to using the Perl
interpreter's lexer. I wrote the first versions and then other contributors
extended the lexer into some of the more difficult pieces of Perl syntax.

   The implementation of lexing and parsing in the Perl interpreter is a bit
messy, using parse level information to guide lexical analysis.

   Neil