<div dir="ltr"><div class="markdown-here-wrapper" id="markdown-here-wrapper-92123" style><p style="margin:1.2em 0px!important">On Fri, May 2, 2014 at 9:04 AM, Doug Blank <<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>> wrote:</p>
<p style="margin:1.2em 0px!important"></p><div class="markdown-here-exclude"><p></p><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div class="">On Thu, May 1, 2014 at 5:35 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">ROUTER sockets use an IDENTITY prefix to determine who the recipient should be. </div>
</blockquote><div><br></div></div><div>Ok, I think I understand, but still not working. Here is some wording that (if correct) could be inserted in an expanded discussion of the wire protocol (which doesn't yet mention socket type):</div>
</div></div></div></blockquote><p></p></div><p style="margin:1.2em 0px!important"></p>
<p style="margin:1.2em 0px!important"></p><div class="markdown-here-exclude"><p></p><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote"><div><a href="http://ipython.org/ipython-doc/dev/development/messaging.html#the-wire-protocol" target="_blank">http://ipython.org/ipython-doc/dev/development/messaging.html#the-wire-protocol</a><br>
</div><div><br></div><div>
"""</div>
<div>If you only want to implement basic functionality (one frontend per kernel, no parallel or cluster support) then using DEALER instead of ROUTER is fine, and a good place to start because you don't need the complexity of the routing protocol. You can simply leave out the routing prefixes before the <IDS|MSG> delimiter in the wire protocol.</div>
<div><br></div><div>If you do want more than the basic functionality (such as allowing two frontends to a kernel) then you must use the ROUTER socket type, and must have the correct routing prefixes.</div><div>"""</div>
</div></div></div></blockquote><p></p></div><p style="margin:1.2em 0px!important"></p>
<p style="margin:1.2em 0px!important">I would not add this. I would consider it purely an accident that any kernel with DEALER sockets sometimes appears to work, and no kernel should ever use DEALER sockets on the kernel. I think suggesting that DEALER sockets might work in simple cases would only increase the likelihood that new kernels use dealer sockets, when none should.</p>
<p style="margin:1.2em 0px!important">The addition of a kernel_info request at startup, which is about to go into IPython master, will mean that no kernel using DEALER sockets will work in any frontend, because there will be a minimum of two client sockets connected to the kernel’s shell channel.</p>
<p style="margin:1.2em 0px!important"></p><div class="markdown-here-exclude"><p></p><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><br></div><div>The problem is that I haven't gotten the routing to work, and it seems like it deserves some documentation. For example, watching the IPython kernel, I appear to see the following kinds of routing:</div>
<div><br></div><div>message type: "kernel_info_reply", routing appears to be "SESSION_ID":</div><div><br></div><div><div>*** KERNEL SENT: "['860e49d1-3852-4d9a-a623-01271cc95f21', '<IDS|MSG>', '1832e7dad75f76550782d188b01d902c259da4b1d12ad9aa51c11bb756176065', '{"date":"2014-05-02T11:36:54.231974","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"e4cc180a-d262-4ac8-b4e0-fbc9e49855fb","msg_type":"kernel_info_reply"}', '{"date":"2014-05-02T11:36:53.956564","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"08428883-db50-401d-8601-e62fadcf38ba","msg_type":"kernel_info_request"}', '{}', '{"protocol_version":[4,1],"language_version":[2,7,5],"language":"python","ipython_version":[2,0,0,""]}']"</div>
</div><div><br></div><div><div>message type: "status", routing appears to be "kernel.KERNEL_ID.status":</div></div><div><br></div><div><div>*** KERNEL SENT: "['kernel.a5092211-ccf1-4017-bab9-58bab7d8eb52.status', '<IDS|MSG>', '1a3ac6b69c87a4e5c30ea95001c86868ee1af08b4e6d953f0478f117ac6f6124', '{"date":"2014-05-02T11:37:56.826835","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"fefb1c9c-8c93-4a7e-85a0-6440eda8af99","msg_type":"status"}', '{"date":"2014-05-02T11:37:56.824306","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"335597c6-2101-4343-8d64-2a9d328617ef","msg_type":"execute_request"}', '{}', '{"execution_state":"busy"}']"</div>
</div><div><br></div><div>message type: "execute_repl", routing is "execute_reply", raw data (end of message) is list of session ids:</div><div><br></div><div><div>*** KERNEL SENT: "[u'execute_reply', {'status': u'ok', 'execution_count': 2, 'user_variables': {}, 'payload': [], 'user_expressions': {}}, {'parent_header': {}, 'msg_type': 'execute_request', 'msg_id': '4a153840-c5f1-4f24-813f-4585c9eb776f', 'content': {'code': '1 + 1\n', 'silent': False, 'user_variables': [], 'store_history': True, 'user_expressions': {}, 'allow_stdin': True}, 'header': {'date': datetime.datetime(2014, 5, 2, 11, 42, 50, 892870), 'username': 'dblank', 'session': '1a402950-a0bf-417d-88d3-13b6d58506c3', 'msg_id': '4a153840-c5f1-4f24-813f-4585c9eb776f', 'msg_type': 'execute_request'}, 'buffers': [], 'metadata': {}}, {'dependencies_met': True, 'engine': u'a5092211-ccf1-4017-bab9-58bab7d8eb52', 'status': u'ok', 'started': datetime.datetime(2014, 5, 2, 11, 42, 50, 896000)}, ['1a402950-a0bf-417d-88d3-13b6d58506c3']]"</div>
</div><div><br></div><div>Am I reading this correctly? If so, then many kernel developers may stop at basic communication and use DEALER (which is fine, I guess). For those of us you want to implement the full spec (with the expectation of having parallel engines working), each of these cases will need to be dealt with, right?</div>
</div></div></div></blockquote><p></p></div><p style="margin:1.2em 0px!important"></p>
<p style="margin:1.2em 0px!important">I suppose I should add a bit of a zmq primer to the message spec. The relevant bits:</p>
<p style="margin:1.2em 0px!important">ROUTER sockets use IDENTITIES to pick the peer to send messages to. This is the part of the message preceding the ‘‘ delimiter. A ROUTER <em>must</em> send replies with the same routing prefix as the requests it receives.</p>
<p style="margin:1.2em 0px!important">A simple (Python) example:</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;overflow:auto;margin:1.2em 0px"><code class="language-python" style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:nowrap;border:1px solid rgb(234,234,234);border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;border-top-left-radius:3px;border-top-right-radius:3px;border-bottom-right-radius:3px;border-bottom-left-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block;padding:0.5em;color:rgb(51,51,51);background:rgb(248,248,255)">DELIM = <span class="string" style="color:rgb(221,17,68)">b'<IDS|MSG>'</span>
<span class="comment" style="color:rgb(153,153,136);font-style:italic"># request_msg normally constructed by router_socket.recv_multipart()</span>
request_msg = [ <span class="string" style="color:rgb(221,17,68)">b'blob'</span>, <span class="string" style="color:rgb(221,17,68)">b'other blob'</span>, DELIM, <span class="string" style="color:rgb(221,17,68)">b'message frame 0'</span>, <span class="string" style="color:rgb(221,17,68)">b'message frame 1'</span>]
delim_idx = request_msg.index(DELIM)
identities = msg[:delim_idx]
msg_frames = msg[delim_idx + <span class="number" style="color:rgb(0,153,153)">1</span>:]
<span class="comment" style="color:rgb(153,153,136);font-style:italic"># handle msg....</span>
<span class="comment" style="color:rgb(153,153,136);font-style:italic"># build reply_frames with the same routing prefix as the request:</span>
reply_frames = [ <span class="string" style="color:rgb(221,17,68)">b'reply 0'</span>, <span class="string" style="color:rgb(221,17,68)">b'reply 1'</span> ]
reply_msg = identities + [DELIM] + reply_frames
router_socket.send_multipart(reply_msg)
</code></pre>
<p style="margin:1.2em 0px!important">PUB sockets are different. PUB sockets use only the first message part as a TOPIC, on which sub sockets can filter. In IPython, this is generally ignored, but the message spec describes what IPython does with pub topics. The PUB topic should <em>not</em> be the routing prefix used by the ROUTER socket.</p>
<p style="margin:1.2em 0px!important">-MinRK</p>
<p style="margin:1.2em 0px!important"></p><div class="markdown-here-exclude"><p></p><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div><br></div><div>Thanks again!</div><span class=""><font color="#888888"><div><br></div><div>-Doug</div></font></span><div><div class="h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
In an IPython message, this would be the frames prior to the '<IDS|MSG>' delimiter. You must send these as the beginning of the reply in order for the reply to arrive at the correct peer.<span><font color="#888888"><div>
<br></div><div>-MinRK</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 1, 2014 at 1:24 PM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>On Thu, May 1, 2014 at 4:01 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">All of those DEALER sockets should be ROUTER sockets. The DEALER behavior is correct for only one peer, but it will round-robin replies if there is more than one.</div>
</blockquote><div><br></div></div><div>If I change the ShellChannel to ROUTER then the kernel will no longer respond.</div><div><br></div><div>If I change the others to ROUTER, then I don't see a difference with prior behavior.</div>
<div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"> Clearly some documentation is wrong. Where did you get the suggestion that they should be DEALER sockets?</div>
</blockquote><div><br></div></div><div>Probably a combination of finding what would work, and looking at other code (it is sometimes hard to translate code and docs between different ZMQ libraries).</div><div><br></div><div>
Thank you for helping me track this down!</div><span><font color="#888888">
<div><br></div><div>-Doug</div></font></span><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><span><font color="#888888"><div>
<br></div><div>-MinRK</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 1, 2014 at 12:45 PM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div>On Thu, May 1, 2014 at 2:37 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">What are your socket types on the Kernel side?</div>
</blockquote><div><br></div></div><div>Here is the definitions for our channels:</div><div><br></div><div> <span style="white-space:pre-wrap">public class ShellChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {</div><div> }</div><div> }</div><span style="white-space:pre-wrap"> public class IOPubChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.PUB) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class ControlChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class StdInChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class HeartBeatChannel : Channel {</span> <div>
<span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.REP) {<br></div><div><div> }</div><div> }</div></div></div><div><br></div><div>Could that really effect multi-frontend, but not single?</div>
<span><font color="#888888">
<div><br></div><div>-Doug</div></font></span><div><div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Thu, May 1, 2014 at 9:58 AM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div dir="ltr">Devs, I have been trying to find the cause of this bug for some time, and I thought that someone might be able to provide a hint as to where to look to find the issue.<div>
<br></div><div>We have an external kernel that works very well when you connect onto it with a single frontend. But when you connect two frontends (console or notebook), then the conversation gets confused.</div>
<div><br></div><div>More detail:</div><div><br></div><div>Terminal 1:</div><div><br></div><div>ipython console --profile calico</div><div><br></div><div>works fine.</div><div><br></div><div>Terminal 2:</div><div><br></div>
<div><div>ipython console --profile calico --existing</div></div><div><br></div><div>starts, but now things are confused. Now, each terminal takes turns being able to get the prompt. If run in notebook, then code execution never signals the end of computing (the [*] never turns to a number, but you can still enter and get results back). In the console, output is received, but the prompt is in the other terminal.</div>
<div><br></div><div>It works fine with a single front end.</div><div><br></div><div>Any ideas appreciated!</div><span><font color="#888888"><div><br></div><div>-Doug</div></font></span></div>
<br></div></div>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote><p></p></div><p style="margin:1.2em 0px!important"></p>
</div></div>