[IPython-dev] Qt/Curses interfaces future: results of the weekend mini-sprint (or having fun with 0mq)

[Fernando Perez]

On Wed, Mar 24, 2010 at 9:56 AM, Brian Granger <ellisonbg at gmail.com> wrote:

>> - We probably want a second REQ/XREP socket used strictly for control
>> messages.  This will  make it easier to handle them separate from code
>> execution.
>
> I think we actually need to have the control messages over the same
> socket as execute.  The reason is that if we had a 2nd channel, the
> control messages could overtake the execute ones:
>
>>>> time.sleep(10);
>>>> a = 10
>>>> a.[TAB]  # if we have a 2nd channel, this will get to the kernel before a = 10!!!
>
> I have some ideas though on how we can better use the single XREQ/XREP
> pair for both control and execution.

With good use of the status socket the above shouldn't happen, as
control requests shouldn't be posted by clients if the kernel is busy,
I think.  But in general, I do agree that we'll probably better off
with a single channel for execution and one for publication, the
proliferation of sockets isn't a good thing.  I think the status and
control 'channels' (not sockets) are needed though, we just need to
have a nice api to manage the messaging on these channels that makes
it not too confusing in practice.  I'm pretty sure it's quite doable,
from the experience so far.

>> - The kernel should also  have  a second PUB socket where it simply
>> posts busy/ready status updates.  This can then be used by clients to
>> check before making certain control requests like tab completion that
>> should be avoided when busy (I know, there's a race condition if it's
>> implemented naively, but I think it can be avoided simply by assuming
>> that control requests are made only when the status socket is in
>> 'ready' status, but that clients can't assume they will get them
>> honored, they have to  check the result and be ready to time out if
>> needed).
>
> Nice idea to have the status updates published.  We should definitely
> do that.  I think we can easily do this using a single PUB/SUB pair
> though.  I just need to write down these ideas I have about how to
> handle multiple types of actions on a single socket.  Shouldn't be a
> problem though.  I am a little weary of having too many open sockets
> and there is really no reason we can't handle all the actions on a
> single socket.

Fully agreed.

>> - We're starting to see the architecture needed for qt/wx/curses
>> applications now: we should break what we now call the 'frontend' into
>> 2 objects:
>>
>> 1. 'Client': object that talks to kernel with zmq messages, does NOT
>> talk directly to user and doesn't know if it's in qt, wx, curses or
>> terminal.
>>
>> 2. 'Frontend': object that talks to user, has UI dependencies (qt,
>> readline, etc) but does NOT have zmq dependencies.  It *only* talks to
>> client object via python calls,  it does not do messaging.
>>
>> Even the code in frontend.py today is starting to have a bit of this,
>> now we just have to make the split, and that will quickly indicate
>> where the design divisions need to go.
>
> I know this is something you really want to have.  But I don't think
> it is possible, even for the synchronous line based frontend.  This is
> because all frontends will need to have an event loop and the event
> loop itself needs handle the 0MQ messaging stuff.  But I am willing to
> explore this idea further to see if it is possible.  I think the next
> step is to implement a real event loop in pyzmq and then use in for
> our current frontend/kernel prototype.  That will better show us what
> the abstractions and interfaces are.

Let's finish up the messaging until we're happy and we'll see if this
is doable or not in practice.  I trust your intuition and you may be
right, though we should still strive to centralize as much as possible
in one common api that all frontends can reuse, to minimize code
duplication.

I'll try to work a bit on this again tomorrow.

Cheers,

f