[IPython-dev] Status of IPython+GUI+Threads+PyOS_InputHook

Gael Varoquaux gael.varoquaux at normalesup.org
Sun Feb 8 15:53:54 EST 2009

On Sun, Feb 08, 2009 at 12:15:53PM -0800, Brian Granger wrote:
> The things you have done to get this working (in my mind) are simply
> welding the Core to wx.  If we have to do things like this for each GUI
> toolkit, we are back to having a core that is non-reusable and
> difficult to maintain, test, extend, debug, etc. 

I am actually not too worried about this. I believe the problems can be
abstracted to a small set of well-defined methods to be implemented by a

> To make matters worse, these types of hacks are notoriously fragile.

Indeed, this worries me a lot/

> The killer though is that, at the end of the day, if a user runs a
> long computation (let's say a big PyMC computation for example), the
> entire GUI will become unresponsive.  To me, the above hacks seems
> like an awful lot of work to get a solution that is still
> sub-standard.

> I think all of us have a vision that
> IPython/numpy/scipy/matplotlib/sympy/ets/etc will allow us to build
> scientific computing environments that blow the doors off of
> Mathematica, Matlab, Maple, IDL, etc.  However, if our software always
> has to be explained with "oh, that, yeh, when you run that type of
> code, the GUI will freeze," we will never reach that goal.

> So, let's try to figure out a real solution to these problems that
> isn't crippled and let's us write robust, reusable and hackable code.

I have given a lot of thought to this, and I believe that to avoid
freezing, there is no other option than multiple processes. If you think
of all the interactive environments that are user friendly, they all run
code in a sandbox that has nothing to do with the GUI (sage included). If
you don't go multi-processes, you are bound to expose some of the
oddities of the event loop, or worst, threads, to the user.

The problem with multiple processes is that if I want to do interactive
data visualization or debugging, I end up copying my data between
processes, and this is not good at all. Maybe the shared array that we
have been discussing on the scipy-users mailing list is an answer to this
problem, but it is not there yet, and it seems like a sledge hammer

If you only want a friendly, interactive terminal that does not freeze
when you run computations, ie an IPython++, I believe something like the
sage approach is excellent. In other words, you have a canvas where you
lay out commands and their output, that lives in a totally different
process than the execution engine. Not too much data flows between the
canvas and the execution engine, so there is no shared memory problem.
Interactive visualization can be done by running a GUI loop in the
execution engine, and having things like matplotlib or Mayavi living in
the same process. I believe the best approach so far would be to build
upon something like knoboo (http://knoboo.com/, which is unfortunatey
GPL) for the front, and an execution engine with an eventloop built upon
IPython1. Maybe the frontend could be made to look cooler by building a
GUI around an embedded Webkit, but that's probably the easiest part.

Interactive visualization of very large dataset is an important feature
for my current work. Mayavi + ipython is fantastic in this regard (sorry
for my lack of humility). It is a great debug tool to be able to
visualize the very exact huge datasets that you are working with, and the
larger the dataset, the more important it is to have interactivity, to be
able to explore it better.

These are the conclusions I came to at the end of my work at Enthought
this summer. The experience I gained from developing the frontend,
integrating it in Mayavi, and developing SciPylab (a scientific
environment integrating all that, that we have been quite quiet about
because it is far from being ready), changed a lot my initial vision of
the problem, I must admit.

My 2 cents,


More information about the IPython-dev mailing list