[Python-ideas] Crazy idea: allow keywords as names in certain positions

Tim Peters tim.peters at gmail.com
Mon May 14 15:38:48 EDT 2018

[Terry Reedy <tjreedy at udel.edu>]
> ...
> The impact on real-time syntax coloring should be considered.  An re-based
> colorizer, like IDLE's, tags every appearance of keywords outside of strings
> and comments, syntactically correct or not. For instance, the first two
> 'and's in "a and b.and # and error" are tagged. To not tag the second would
> require full parsing, presumably by compiling to AST, and more support code.
> I suspect this would would make coloring slower.  Whether too slow for
> existing machines, I don't know.

I can pretty much guarantee it would be slower - way slower.

But regexps could still do the bulk of it, provided the contexts
remain as simple-minded as the specific ones Guido suggested:

    For example, we could allow keywords after 'def' and after a period,

So when the IDLE colorizer regexp finds a match with the "KEYWORD"
group name, don't color it at once.  Instead look back from the start
of the _possible_ keyword match, to see whether it's preceded by

     period maybe_whitespace
     "def"-starting-at-a-word-boundary  at_least_one_whitespace

and if so skip coloring.

There are many ways to code that, but - ya - they're annoying.  Alas,
the re module's negative lookbehind assertion isn't powerful enough
for it, else those checks could be added directly to the keyword part
of the colorizing regexp.  I expect (but don't know) that lookbehinds
in MRAB's "regex" module are strong enough for it.

Of course that's just IDLE.  It was remarkably easy to teach my
primary editor (Source Insight) all sorts of stuff about .py files,
but its "keywords" subsystem is restricted to specifying literal
strings to look for.  The kinds of tricks above probably require that
the tool have access to a real programming language (like IDLE and
Emacs enjoy).

More information about the Python-ideas mailing list