[IPython-dev] messaging protocol

[MinRK]

On Thu, Apr 7, 2011 at 18:27, Jason Grout <jason-sage at creativetrax.com>wrote:

> I've been pondering the messaging spec quite a bit over the last few
> weeks and meshing it in with a project in Sage (a single-cell server,
> like a one-off ipython or sage web notebook).  Our architecture is that
> we have a big database sitting on the server side that basically acts as
> a cache of the zeromq messages between the kernel and the web client. It
> is easy enough to store the messages in the database, but we have a
> problem that you guys don't have since you use zeromq end-to-end.  Once
> we put a list of messages into the database, we don't know what the
> right order for the messages is.  So here is a
> suggestion/request/proposal for the API:
>
> PROPOSAL: Can we add another field to the header of a, a msg_order (or
> msg_counter?) field, which (across messages with the same
> header['session']) is guaranteed to be an increasing integer signifying
> the order of messages?
>

The parallel code uses a slightly more advanced Session object (the thing
that builds messages) that includes the notion of an extensible 'subheader'.
With this, you can put any extra (jsonable) information you want into the
header.


>
> You might ask why we don't just use the msg_id field for this.  Well, it
> seems nice for us to make the msg_id be a mongodb id object, which is
> just a hash.  It seems elegant to let the client or server dictate what
> how the msg_id's are formed with no insistence on a particular format or
> structure for the msg_id, other than it be unique among messages in the
> same session.


The Client's Session object (or whoever builds the message) creates the
msg_id,
and the only restriction is that it be jsonable.  The Client building the
message
can use any scheme they like to build msg_ids, same with the Server and its
replies,
and they don't even have to be the same as each other.

 In fact, I'm not even sure there's anywhere in the code that actually
requires that msg_id be unique
(though *we* do, for various reasons).


>
> You must have had a related problem storing history, though.  How did
> you sort the messages in the history database?
>

the IPython console has an execution_count (the number that determines what
goes in 'In [nn]'.  the sqlite database stores the session ID and execution
counter for every entry.


>
> On the other hand, since the proposed msg_order *is* guaranteed to be
> unique across all messages in a session, maybe there's nothing lost in
> (just us?) insisting that the msg_id be an increasing integer sequence.
>

Originally, the msg_id was just an integer, but that was replaced with UUIDs
to
allow for simultaneous clients.  If you don't have more than one client per
kernel,
there is no reason to not use integers for msg_id.

-MinRK


>
> Thanks,
>
> Jason
>
> --
> Jason Grout
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110407/87e0ea51/attachment.html>