[Python-ideas] AST Transformation Hooks for Domain Specific Languages

[Eugene Toder]

>> My understanding is that the AST transformation is done at compile-time,
>> while the decorator happens at run-time.  So your .pyc file gets your
>> transformed code.
>
>Yep - it's the move to compile time metaprogramming that makes ideas
>like this one and tools like Mython interesting.
In python compile time and run time are less separated than in many
other languages. I don't see a big difference between doing
transformation at compile time proper vs. at import time (except for
start-up time, if the transformation is taking seconds). Technically,
decorator that parses docstring is as powerful as an AST
transformation. The difference is mostly about end user friendliness
and aesthetics -- whether you feel like you're using a feature or a
cheap hack.

> So a simple "python -mdsl.sql myfile.py" would run a file that uses
> the DSL, while "python -i -mdsl.sql" would get you an interactive
> interpreter that understood that DSL dialect.
Ok, it's easy to do, but to me the fact that I have to even think
about it makes it less convenient.

> Reusing Python syntax would still be easy - you'd simply invoke
> ast.parse() on the stringified body.
Right, otherwise I would knew that I *don't* like the idea. At the
moment I don't know if I like it :)
Using Python parser first leads to a slippery slope: Python syntax is
only tuned for one language -- Python. If we start using it as a base
for DSLs more widely, we'll want some generic extensions, which would
give syntax error in python, but would produce AST nodes for DSLs to
transform. E.g. space delimited expression list ('select foo') or
custom suite ('join:\nleft\nright\ncond').
Stringifying function body avoids this problem (though one would have
to write a full parser even if we wants a very little tweak to python
syntax), but it allows completely non-python syntax mixed with python.
I don't know if it's a big problem, though.

Eugene