[IPython-dev] Comments on IPython Qt widgets

[Fernando Perez]

Dear Charlie,

Just joining the others to welcome you to the list!  It's great to see
interest and new participants with regards to the new architecture and
the Qt machinery. I hope we'll be able to help yo contribute and do
useful thins with ipython in the GIS domain.

On Mon, Feb 7, 2011 at 6:52 PM, Charlie Sharpsteen
<source at sharpsteen.net> wrote:
> Hello IPython developers,
> I was playing around with the new Qt  widgets in 0.11-dev the other day and
> was very impressed.  I tried throwing together a quick plugin to add an
> IPython console to the Qt-based program Quantum GIS [QGIS][1].  Creating the
> plugin was easy enough as QGIS runs an embedded Python interpreter and
> exposes a lot of PyQt4 functionality.  However, I noticed a couple of things
> and was wondering if anyone on this list had any comments to share:
>

> - Currently, the only IPython kernel available for use with the Qt widget
> runs the IPython interpreter in a separate process.  For QGIS, it would be
> very nice to have the option of running an IPython kernel inside the same
> process as the Qt widgets.  IPython would then have direct access to the
> QGIS interface and would be able to inspect and operatie on the PyQt4
> objects that compose the GUI providing a powerful tool for users performing
> spatial analysis and developers building plugins.
> Are there any plans to develop IPython kernels that do not run
> in separate processes?  I spent a few days trying to build one myself by
> wrapping an InteractiveShell, but I feel like I am flailing a little bit
> when it comes to creating a kernel_manager that will integrate with the Qt
> frontend.  If anyone has any design advice on how to implement an in-process
> kernel I would love to hear it.

I actually think that this is something we do very much need and would
like to have, despite the (absolutely valid) caveats that Brian
raised.  While it's true that an in-process kernel can block and/or
crash your entire app, it can also be very useful (see Mayavi for a
good example where even with rather gross hack remains extremely
useful).

Let me try to explain our architecture a little bit.  If this is clear
enough let me know, and I'll add it to our dev docs.  For all of this,
keep this diagram in mind:

http://ipython.github.com/ipython-doc/dev/development/messaging.html

In a two-process model, the Kernel is proxied on the client process by
a KernelManager object.  This is the object that all client code uses
to communicate with the kernel, and it's the *only* object in charge
of doing explicit zmq calls.  The rest of ipython has no idea zmq
exists, it just talks to this KernelManager object.  The basic
KernelManager lives in:

https://github.com/ipython/ipython/blob/master/IPython/zmq/kernelmanager.py

but that can't be used directly, since each front end needs to have
code to specifically handle the generated messages according to its
own UI constraints. So for example the Qt console has here its version
that wires the message handlers to Qt's event mechanisms:

https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/kernelmanager.py

With this in mind, what would be needed is an object that exposes the
kernelmanager interface but runs in-process instead of out of process,
and is otherwise the same.  Then, we could compose for Qt two KM's:
one based on an out-of-process, zmq-based class (what we have today)
and another one based on such an in-process KM that doesn't use zmq.

In this manner, the code above would function identically, modulo some
modifications so it doesn't mention external kernels on shutdown and
somesuch.

Let us know if this makes sense, I think it would be great to get
something like this implemented in full.

> Keep up the good work on IPython development!

Thanks for the kind words!   We hope to have you on board for the
project.  The new architecture is actually what we think is a fairly
good and clean design, so contributions like what you're discussing
would be great to have.

Regards,

f