[Python-ideas] AST Transformation Hooks for Domain Specific Languages

[Eugene Toder]

> Yep, but if you're at the point of using a DSL, that's likely to be
> part of a larger framework which can take care of registering the DSL
> for you before you import any modules that need it.

Not necessarily. And registering makes debugging small pieces and
playing in interactive interpreter less convenient.

> Yeah, the big downside is having to almost completely reimplement the
> AST compiler in order to do it that way (since the built-in one would
> choke on the syntax extension).

I did not realize you're proposing to skip parsing of the body of dsl
function and just tuck all the code into Str literal.
I though the idea was to actually do an AST transformation, i.e. use
Python parser first and rewrite resulting AST into something else.
This is a classical approach and something people already do in
Python, though it's not integrated with the language at the moment
(e.g. there's no quote and no way to get AST for function).
If the idea is to use completely custom syntax in dsl function and
implement custom parsers for dsls, it's harder to do with import hook,
but not too hard. A preprocessor step that puts body of dsl functions
into """ will do.

I don't know if I like the stringification idea. It allows much more
freedom in dsl syntax, but to use that freedom one needs to implement
a parser. Just rewriting AST is simpler. Also, the implementation is
very different from an AST transform -- basically, the grammar needs
to say that after you saw dsl function header the next suite needs to
be saved as a string.

> But what would the eager decorator buy you over just specifying a
> different dialect in the "from" clause?

The point of eager decorator is to avoid the need for registration --
code becomes self-sufficient. It doesn't need "from" clause, I forgot
to delete it. To reiterate, the idea is that the code in form
(straw-man syntax)

@@a.dec(args)
def foo():
   ...

in module b will import module a at compile time and execute a.dec on
foo's AST and resulting AST will be used.
This can be done with a 'from' clause as well, if it gains importing
capabilities -- that's just a syntactic difference. However, this
seems very close to existing decorators, so similar syntax and
semantics seem to make sense.

Eugene