[IPython-dev] Message spec draft more fleshed out

[Fernando Perez]

Hey Min,

On Tue, Aug 10, 2010 at 11:55 AM, MinRK <benjaminrk at gmail.com> wrote:
> This is great.
> There are a few additional functionalities I need on top of this, that I
> have added to the message spec I use in my parallel code.
> I have multiple clients,and need unique message ids, so clearly ints are
> inadequate.  I switched msg_id to also be a uuid. I could certainly generate
> unique msg ids in the controller by combining the msg id and the session id,
> which should be unique.

OK, should we just switch to these right away?  If so, does this sound
like the right way to make them: ?

uuid.uuid1(os.getpid())

We'd obviously cache the pid, but this lets us seed the uuid1 call
with the pid of eac client.  Alternatively we can call uuid4(), and
trust that the probability of collisions is low enough. I sort of like
better the idea of seeding with a known quantity; we could combine
hostid and pid if we want to be extra safe, but I don't think it's
worth worrying about that level of low-probability of collision, is
it?

> Since I need to inspect messages on the way, and don't want to have to
> unpack the content of the message, I can't send the whole message as one
> json object. For this, I split it, such that the headers and content are
> sent separately. msg_type is added to the header for this.

Should we list the multipart spec separately?  This would be only for
'data carrying' messages, or would it be for all communications?

> I need to be able to send data without copying, and for that I added a
> 'buffers' element at the top level of a message.  I also added an
> apply_message type, for using Brian's apply model. I will write up how the
> apply stuff works later (I expect there will be some discussion and
> rearrangement of some of it).
> I also added, but no longer use, a subheader, which allows senders of
> messages to extend the header.  I needed this when the Controller parses a
> message destined for an engine, it shouldn't unpack the content of the
> message, only the header. Since the routing is now handled purely in zmq, I
> don't currently have a need for the subheader, but I can certainly imagine
> it being useful.  This is not so much a part of the root message format as
> it is a part of the session.send() api.

I'm finishing up the doc, it would be good if you could write up these
ideas into it so we have all the design in one place...  I'll ping
soon with the finished draft.

Cheers,

f