[IPython-dev] ipython in a background thread

Zachary Pincus zachary.pincus at yale.edu
Mon May 30 15:46:05 EDT 2011


Thanks for the pointers into the code, Fernando! I saw that there were a lot of changes in 0.11, but I wasn't sure where to go to see what the new mechanics were. Most obliged.

Zach


On May 28, 2011, at 5:18 PM, Fernando Perez wrote:

> Hi Zach,
> 
> On Fri, Apr 29, 2011 at 10:03 AM, Zachary Pincus
> <zachary.pincus at yale.edu> wrote:
>> I find that ipython, specifically ipython running in a good terminal (as opposed to embedded in a GUI window), provides a really great interface (e.g. in my case, a microscope and camera hardware). Given this use, it's been important to also have a GUI thread running so I can throw acquired images rapidly up onto a GL canvas or something. For a while, I've used a hacked-up pyglet runloop that would work in a background thread, and set up a simple message-passing system.
>> 
>> This got quite kluged quite fast though, and it turns out this approach won't work at all with cocoa GUIs on OS X, which seem to require running on thread-0 (a pity). I know people on this list have looked and thought about these issues a lot, so I'd be curious what seems like the best approach in general? (There used to be some code in IPython for doing this with various window toolkits, but I don't see that in the 0.10.2's codebase anymore...)
>> 
>> - I could run the GUI from thread-0 and try message-passing to IPython on another thread. Does this work well at all, or is it a huge kluge to get all the readline etc. features working right?
>> 
> 
> That's what we had always in the 0.10.x series, and it's horribly
> brittle.  In master/0.11, we have compelely restructured how GUI event
> loop integration is handled, to work instead with PyOSInputHook and
> tie directly into the event loop of each toolkit.  There's already
> support for Qt, WX and GTK (Tk works automatically because Python does
> the same internally).  It should be possible to do something similar
> for OpenGL/pyglet, I think.
> 
> Here are some pointers in the documentation:
> 
> http://ipython.github.com/ipython-doc/dev/interactive/reference.html#gui-event-loop-support-support
> 
> The implementation machinery lives here:
> 
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py
> https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookgtk.py
> 
> And these are the key parts of the code that use it:
> 
> ./frontend/terminal/interactiveshell.py:
>   27 : from IPython.lib.inputhook import enable_gui
> ./frontend/terminal/ipapp.py:
>   39 : from IPython.lib import inputhook
>  513 :             gui = inputhook.GUI_WX
>  515 :             gui = inputhook.GUI_QT
>  517 :             gui = inputhook.GUI_GTK
>  531 :             activate = inputhook.enable_gui
> 
> It would be great to have similar GL integration as we have for the
> others, so please let us know if any of this makes sense, and if you
> make any progress we'd love to integrate it.
> 
>> - I could run the GUI in a separate process entirely, which would force a much cleaner API, but I'm not sure if pumping images at video-rate could be done cleanly. Maybe with shared memmapped arrays? (I've seen some recipes for this sort of thing on the numpy list lately.)
>> 
> 
> If the client is only going to consume the data read-only (or writing
> to it doesn't matter anymore for the sender), then you may consider
> our two-process architecture, used by the Qt console.  It will require
> some more design, that's true, but you'll have something very robust
> and flexible in general, with possibilities for multi-client
> monitoring, remote monitoring, etc.  The pyzmq code we use has the
> ability to do zero-copy sends of numpy arrays and is quite fast, so it
> might be possible to obtain video frame rates.
> 
> Something like that would be a very cool application of this
> architecture, let us know if you decide to go down that road and we
> might be able to provide some advice.
> 
> Best,
> 
> f




More information about the IPython-dev mailing list