[IPython-dev] ipython in a background thread

Fernando Perez fperez.net at gmail.com
Sat May 28 17:18:45 EDT 2011


Hi Zach,

On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
<zachary.pincus at yale.edu> wrote:
> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>
> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>
> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>

That's what we had always in the 0.10.x series, and it's horribly
brittle.  In master/0.11, we have compelely restructured how GUI event
loop integration is handled, to work instead with PyOSInputHook and
tie directly into the event loop of each toolkit.  There's already
support for Qt, WX and GTK (Tk works automatically because Python does
the same internally).  It should be possible to do something similar
for OpenGL/pyglet, I think.

Here are some pointers in the documentation:

http://ipython.github.com/ipython-doc/dev/interactive/reference.html#gui-event-loop-support-support

The implementation machinery lives here:

https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py
https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py
https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookgtk.py

And these are the key parts of the code that use it:

./frontend/terminal/interactiveshell.py:
   27 : from IPython.lib.inputhook import enable_gui
./frontend/terminal/ipapp.py:
   39 : from IPython.lib import inputhook
  513 :             gui = inputhook.GUI_WX
  515 :             gui = inputhook.GUI_QT
  517 :             gui = inputhook.GUI_GTK
  531 :             activate = inputhook.enable_gui

It would be great to have similar GL integration as we have for the
others, so please let us know if any of this makes sense, and if you
make any progress we'd love to integrate it.

> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>

If the client is only going to consume the data read-only (or writing
to it doesn't matter anymore for the sender), then you may consider
our two-process architecture, used by the Qt console.  It will require
some more design, that's true, but you'll have something very robust
and flexible in general, with possibilities for multi-client
monitoring, remote monitoring, etc.  The pyzmq code we use has the
ability to do zero-copy sends of numpy arrays and is quite fast, so it
might be possible to obtain video frame rates.

Something like that would be a very cool application of this
architecture, let us know if you decide to go down that road and we
might be able to provide some advice.

Best,

f



More information about the IPython-dev mailing list