[IPython-dev] Input transformation rework - input requested

[Aaron Meurer]

I'm sure you're already aware, but we use them for SymPy for some
(optional) transformers that make Python more symbolic friendly
(auto-replace undefined variables with Symbols, wrap integer literals
with Integer so that exact rational numbers work, and so on). The AST
transformer is nice but limiting. We can't use it to wrap float
literals with higher precision for instance because Python drops the
precision of floats in the AST.

There are already some open issues about some limitations we've seen
(https://github.com/sympy/sympy/issues/14440 and
https://github.com/ipython/ipython/issues/10893). I'm not fully
updated on the specifics of what the limitations were, but from what I
remember, there were issues with IPython doing things line-by-line.

I don't know how the new code is architected, but it would be nice if
the transformers could just access the raw input from IPython, and
various transformers could be plugged in to that to make things
simpler if desired (the AST transformer could be one such instance).

At the very least, if you want to test the new design, you could look
at translating our transformers in SymPy (including the work in
progress float transformer at
https://github.com/sympy/sympy/pull/13300).

Aaron Meurer

On Sun, May 27, 2018 at 2:50 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Thomas this is great news! I think that part of the code base could benefit
> from being simpler and more extensible. Thanks for tackling this!
>
> Sent from my iPhone
>
> On May 27, 2018, at 1:19 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> Hi all,
>
> For IPython 7, I'm planning a rework of the input transformation framework:
> https://github.com/ipython/ipython/pull/11041
>
> The new framework is - I hope - simpler than the one it replaces, and will
> hopefully have fewer weird corner cases. But it's still a pretty complex
> beast, and I don't feel like it's a good platform for people to add more
> transformations to. One option to improve this is to make a smaller
> extensible API, restricting the kind of things that third party code can do.
>
> I don't think there's ever been much third party code extending IPython's
> input transformation, but I'd like to find out about the code that does. So
> if you're aware of a project that does use our input transformation API:
>
> - What project?
> - What does it want to transform?
> - Can you point me to the code?
> - What fallback options would it have if IPython didn't support the
> transformation it wanted?
>
> This is specifically about text transformations; AST transformations are not
> changed.
>
> Thanks,
> Thomas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at python.org
> https://mail.python.org/mailman/listinfo/ipython-dev
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at python.org
> https://mail.python.org/mailman/listinfo/ipython-dev
>