[IPython-dev] Uniform way of integrating event loops among different IDE's

[Almar Klein]

Hi,

> Thinking of wilder ideas, it might even be possible to share a common
> > interpreter (with which I mean the code running in the second process).
> Such
> > that only the way of controlling it is different. Whether one uses
> IPython
> > or IEP, under the hood it's the same thing. There are of course some
> > fundamental differences between IEP and IPython (for example IEP needs to
> be
> > able to run a selection of code), but who knows?
>
> Not wild at all, in fact all the recent work from the Google SoC
> students, as well as the stuff Brian, Evan and I are currently working
> on, goes precisely in that direction.  We're basically specifying an
> 'IPython protocol' as described here:
>
> http://ipython.scipy.org/doc/nightly/html/development/messaging.html
>
> That messaging spec fully defines how to interact with an ipython
> kernel that has all the goodies we see today in the terminal, and any
> number of frontends can talk to one.  The 'newkernel' branch in the
> ipython repo has a Qt implementation of a client that uses this spec,
> and it's getting to be pretty functional already.  We're doing a ton
> of work on this over the next few weeks.
>

I've read some of the documentation for the new stuff you're working on. It
all sounds really well thought through, and am looking forwards for the
results. I've got a couple of questions though.

- I see the possibilities of distributed computing by connecting multiple
kernels to a single client. However, I don't get why you would want to
connect multiple clients to a single kernel at the same time?

- I saw an example in which you're kind of going towards a Mathematica/Sage
type of UI. Is this what you're really aiming at, or is this one possible
front end? I'm asking because IEP has more of a Matlab kind of UI, with an
editor from which the user can run code (selected lines or cells: code
between two lines starting with two ##'s). Would that be compatible with the
kernel you're designing?

- About the heartbeat thing to detect whether kernels are still alive. I use
a similar concept in the channels module. I actually never realized that
this would fail if Python is running extension code. However, I do run
Cython code that takes about a minute to run without problems. Is that
because it's Cython and the Python interpreter is still involved? I'll do
some test running Cython and C code next week.


Since I think its interesting to see that we've taking rather different
approaches to do (more or less) the same thing, I'll share some background
on what I do in IEP:

I use one Channels instance from the channels.py module, which means all
communication goes over one socket. However, I can use as many as 128
different channels each way. Instead of a messaging format, I use a channel
for each task. By the way, I'm not saying my method is better; yours is
probably more "scalable", mine requires no/little message processing. So
from the kernel's perspective, I have one receiving channel for stdin, two
sending for stdout and stderr, one receiving for control (mostly debugging
at the moment) and one sending for status messages (whether busy/ready, and
debug info). Lastly there's one receiving and one sending channel for
introspection requests and responses.

To receive code, sys.stdin is replaced with a receivingChannel from
channels.py, which is non-blocking. The readline() method (which is what
raw_input() uses) *is* blocking, so that raw_input() behaves appropriately.

The remote process runs an interpreter loop. Each iteration the interpreter
checks (non-blocking) the stdin for a command to be run. If there is, it
does so using almost the same code in code.py. Next (if required) process
GUI events. Next produce prompt if necessary, and send status. In another
thread, there is a loop that listens for introspection requests
(auto-completion, calltips, docs).

This code is all in iepRemote1.py and iepRemote2.py if you're interested in
the details.

Cheers,
  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100828/5af306cc/attachment.html>