[Python-ideas] Enabling access to the AST for Python code

[Ben Hoyt]

Oh wait, macropy already has this exact thing. They call it PINQ
(kinda Python LINQ), and they're macro-compiling it to SQLAlchemy
calls.

https://github.com/lihaoyi/macropy#pinq-to-sqlalchemy

Wow.

-Ben

On Thu, May 21, 2015 at 10:10 PM, Ben Hoyt <benhoyt at gmail.com> wrote:
> Huh, interesting idea. I've never used import hooks. Looks like the
> relevant macropy source code is here:
>
> https://github.com/lihaoyi/macropy/blob/master/macropy/core/import_hooks.py
>
> So basically you would do the following:
>
> 1) intercept the import
> 2) find the source code file yourself and read it
> 3) call ast.parse() on the source string
> 4) do anything you want to the AST, for example turn the "select(c for
> c in Customer if sum(c.orders.price) > 1000" into whatever SQL or
> other function calls
> 5) pass the massaged AST to compile(), execute it and return the module
>
> Hmmm, yeah, I think you're basically suggesting macro-like processing
> of the AST. Pretty cool, but not quite what I was thinking of ... I
> was thinking select() would get an AST object at runtime and do stuff
> with it.
>
> -Ben
>
> On Thu, May 21, 2015 at 9:51 PM, Andrew Barnert <abarnert at yahoo.com> wrote:
>> On May 21, 2015, at 18:18, Ben Hoyt <benhoyt at gmail.com> wrote:
>>>
>>> (I know that there's the "ast" module and ast.parse(), which can give
>>> you an AST given a *source string*, but that's not very convenient
>>> here.)
>>
>> Why not? Python modules are distributed as source. You can pretty easily write an import hook to intercept module loading at the AST level and transform it however you want. Or just use MacroPy, which wraps up all the hard stuff (especially 2.x compatibility) and provides a huge framework of useful tools. What do you want to do that can't be done that way?
>>
>> For many uses, you don't even have to go that far--code objects remember their source file and line number, which you can usually use to retrieve the text and regenerate the AST.