[IPython-dev] Splitting input by AST nodes

[Thomas Kluyver]

Our current methods of splitting input into 'blocks' is...well, one of the
comments starts "HACK!!!". It's also the cause of at least issue 306*, and
trying to change it is likely to lead us into a minefield of other similar
problems.

I propose that we redo this with a system making use of the AST (abstract
syntax tree) capabilities introduced in Python 2.6. Using the ast module, we
can parse a code string into an AST, and then use the nodes from that
instead of code blocks. AST nodes can be compiled into standard Python code
objects to be run with exec. This has a couple of consequences which I've
thought of so far: firstly, there's no easy way to get back the code as a
string from an ast node (third party modules exist, but I get the impression
they're rather hackish themselves). So we could no longer do things like "if
the last node is two lines or less, execute it differently". Secondly, "a =
11; b = 12" is currently understood as one block, but in the AST it would be
two nodes.

Since this would obviously be a fairly major change to a key part of
IPython, I thought I'd get some feedback before I start trying to implement
it. Does using AST nodes instead of blocks sound useful, or is there a good
reason why it wouldn't work?

Thanks,
Thomas


* https://github.com/ipython/ipython/issues/306
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110402/56f88dd2/attachment.html>