[Python-ideas] Enabling access to the AST for Python code

[Steven D'Aprano]

On Thu, May 21, 2015 at 06:51:34PM -0700, Andrew Barnert via Python-ideas wrote:
> On May 21, 2015, at 18:18, Ben Hoyt <benhoyt at gmail.com> wrote:
> > 
> > (I know that there's the "ast" module and ast.parse(), which can give
> > you an AST given a *source string*, but that's not very convenient
> > here.)
> 
> Why not? Python modules are distributed as source.

*Some* Python modules are distributed as source. Don't forget that 
byte-code only modules are officially supported.

Functions may also be constructed dynamically, at runtime. Closures may 
have source code available for them, but functions and methods 
constructed with exec (such as those in namedtuples) do not.

Also, the interactive interpreter is a very powerful tool, but it 
doesn't record the source code of functions you type into it.

So there are at least three examples where the source is not available 
at all. Ben also talks about *convenience*: `func.ast` will always be 
more convenient than:

import ast
import parse
ast.parse(inspect.getsource(func))

not to mention the wastefulness of parsing something which has already 
been parsed before. On the other hand, keeping the ast around even when 
it's not used wastes memory, so this is a classic time/space trade off.


> You can pretty 
> easily write an import hook to intercept module loading at the AST 
> level and transform it however you want. 

Let's have a look at yours then, that ought to only take a minute or 
three :-) 

(That's my definition of "pretty easily".)

I think that the majority of Python programmers have no idea that you 
can even write an import hook at all, let alone how to do it.


-- 
Steve