[IPython-dev] Splitting input by AST nodes

Brian Granger ellisonbg at gmail.com
Sat Apr 2 21:06:53 EDT 2011

Fernando knows the most about this code, but I understood that our
input splitter actually works on IPython code (with magics and !ls,
etc.) not plain Python code.  Thus, we can't use the AST splitter.
But this code is *super* subtle, so let's see what Fernando thinks.



On Sat, Apr 2, 2011 at 3:02 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> Our current methods of splitting input into 'blocks' is...well, one of the
> comments starts "HACK!!!". It's also the cause of at least issue 306*, and
> trying to change it is likely to lead us into a minefield of other similar
> problems.
> I propose that we redo this with a system making use of the AST (abstract
> syntax tree) capabilities introduced in Python 2.6. Using the ast module, we
> can parse a code string into an AST, and then use the nodes from that
> instead of code blocks. AST nodes can be compiled into standard Python code
> objects to be run with exec. This has a couple of consequences which I've
> thought of so far: firstly, there's no easy way to get back the code as a
> string from an ast node (third party modules exist, but I get the impression
> they're rather hackish themselves). So we could no longer do things like "if
> the last node is two lines or less, execute it differently". Secondly, "a =
> 11; b = 12" is currently understood as one block, but in the AST it would be
> two nodes.
> Since this would obviously be a fairly major change to a key part of
> IPython, I thought I'd get some feedback before I start trying to implement
> it. Does using AST nodes instead of blocks sound useful, or is there a good
> reason why it wouldn't work?
> Thanks,
> Thomas
> * https://github.com/ipython/ipython/issues/306
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu and ellisonbg at gmail.com

More information about the IPython-dev mailing list