<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, May 1, 2014 at 5:35 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">ROUTER sockets use an IDENTITY prefix to determine who the recipient should be. </div>
</blockquote><div><br></div><div>Ok, I think I understand, but still not working. Here is some wording that (if correct) could be inserted in an expanded discussion of the wire protocol (which doesn't yet mention socket type):</div>
<div><a href="http://ipython.org/ipython-doc/dev/development/messaging.html#the-wire-protocol" target="_blank">http://ipython.org/ipython-doc/dev/development/messaging.html#the-wire-protocol</a><br></div><div><br></div><div>
"""</div>
<div>If you only want to implement basic functionality (one frontend per kernel, no parallel or cluster support) then using DEALER instead of ROUTER is fine, and a good place to start because you don't need the complexity of the routing protocol. You can simply leave out the routing prefixes before the <IDS|MSG> delimiter in the wire protocol.</div>
<div><br></div><div>If you do want more than the basic functionality (such as allowing two frontends to a kernel) then you must use the ROUTER socket type, and must have the correct routing prefixes.</div><div>"""</div>
<div><br></div><div>The problem is that I haven't gotten the routing to work, and it seems like it deserves some documentation. For example, watching the IPython kernel, I appear to see the following kinds of routing:</div>
<div><br></div><div>message type: "kernel_info_reply", routing appears to be "SESSION_ID":</div><div><br></div><div><div>*** KERNEL SENT: "['860e49d1-3852-4d9a-a623-01271cc95f21', '<IDS|MSG>', '1832e7dad75f76550782d188b01d902c259da4b1d12ad9aa51c11bb756176065', '{"date":"2014-05-02T11:36:54.231974","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"e4cc180a-d262-4ac8-b4e0-fbc9e49855fb","msg_type":"kernel_info_reply"}', '{"date":"2014-05-02T11:36:53.956564","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"08428883-db50-401d-8601-e62fadcf38ba","msg_type":"kernel_info_request"}', '{}', '{"protocol_version":[4,1],"language_version":[2,7,5],"language":"python","ipython_version":[2,0,0,""]}']"</div>
</div><div><br></div><div><div>message type: "status", routing appears to be "kernel.KERNEL_ID.status":</div></div><div><br></div><div><div>*** KERNEL SENT: "['kernel.a5092211-ccf1-4017-bab9-58bab7d8eb52.status', '<IDS|MSG>', '1a3ac6b69c87a4e5c30ea95001c86868ee1af08b4e6d953f0478f117ac6f6124', '{"date":"2014-05-02T11:37:56.826835","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"fefb1c9c-8c93-4a7e-85a0-6440eda8af99","msg_type":"status"}', '{"date":"2014-05-02T11:37:56.824306","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"335597c6-2101-4343-8d64-2a9d328617ef","msg_type":"execute_request"}', '{}', '{"execution_state":"busy"}']"</div>
</div><div><br></div><div>message type: "execute_repl", routing is "execute_reply", raw data (end of message) is list of session ids:</div><div><br></div><div><div>*** KERNEL SENT: "[u'execute_reply', {'status': u'ok', 'execution_count': 2, 'user_variables': {}, 'payload': [], 'user_expressions': {}}, {'parent_header': {}, 'msg_type': 'execute_request', 'msg_id': '4a153840-c5f1-4f24-813f-4585c9eb776f', 'content': {'code': '1 + 1\n', 'silent': False, 'user_variables': [], 'store_history': True, 'user_expressions': {}, 'allow_stdin': True}, 'header': {'date': datetime.datetime(2014, 5, 2, 11, 42, 50, 892870), 'username': 'dblank', 'session': '1a402950-a0bf-417d-88d3-13b6d58506c3', 'msg_id': '4a153840-c5f1-4f24-813f-4585c9eb776f', 'msg_type': 'execute_request'}, 'buffers': [], 'metadata': {}}, {'dependencies_met': True, 'engine': u'a5092211-ccf1-4017-bab9-58bab7d8eb52', 'status': u'ok', 'started': datetime.datetime(2014, 5, 2, 11, 42, 50, 896000)}, ['1a402950-a0bf-417d-88d3-13b6d58506c3']]"</div>
</div><div><br></div><div>Am I reading this correctly? If so, then many kernel developers may stop at basic communication and use DEALER (which is fine, I guess). For those of us you want to implement the full spec (with the expectation of having parallel engines working), each of these cases will need to be dealt with, right?</div>
<div><br></div><div>Thanks again!</div><div><br></div><div>-Doug</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr">
In an IPython message, this would be the frames prior to the '<IDS|MSG>' delimiter. You must send these as the beginning of the reply in order for the reply to arrive at the correct peer.<span><font color="#888888"><div>
<br></div><div>-MinRK</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 1, 2014 at 1:24 PM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">
<div>On Thu, May 1, 2014 at 4:01 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">All of those DEALER sockets should be ROUTER sockets. The DEALER behavior is correct for only one peer, but it will round-robin replies if there is more than one.</div>
</blockquote><div><br></div></div><div>If I change the ShellChannel to ROUTER then the kernel will no longer respond.</div><div><br></div><div>If I change the others to ROUTER, then I don't see a difference with prior behavior.</div>
<div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"> Clearly some documentation is wrong. Where did you get the suggestion that they should be DEALER sockets?</div>
</blockquote><div><br></div></div><div>Probably a combination of finding what would work, and looking at other code (it is sometimes hard to translate code and docs between different ZMQ libraries).</div><div><br></div><div>
Thank you for helping me track this down!</div><span><font color="#888888">
<div><br></div><div>-Doug</div></font></span><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><span><font color="#888888"><div>
<br></div><div>-MinRK</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 1, 2014 at 12:45 PM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div>On Thu, May 1, 2014 at 2:37 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
</div></div></div><div class="gmail_extra"><div class="gmail_quote"><div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr">What are your socket types on the Kernel side?</div>
</blockquote><div><br></div></div><div>Here is the definitions for our channels:</div><div><br></div><div> <span style="white-space:pre-wrap">public class ShellChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {</div><div> }</div><div> }</div><span style="white-space:pre-wrap"> public class IOPubChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.PUB) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class ControlChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class StdInChannel : Channel {
</span><div><span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.DEALER) {<br></div><div> }</div><div> }</div><span style="white-space:pre-wrap"></span><span style="white-space:pre-wrap"> public class HeartBeatChannel : Channel {</span> <div>
<span style="white-space:pre-wrap"> </span> base(session, auth, transport, address, port, SocketType.REP) {<br></div><div><div> }</div><div> }</div></div></div><div><br></div><div>Could that really effect multi-frontend, but not single?</div>
<span><font color="#888888">
<div><br></div><div>-Doug</div></font></span><div><div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Thu, May 1, 2014 at 9:58 AM, Doug Blank <span dir="ltr"><<a href="mailto:doug.blank@gmail.com" target="_blank">doug.blank@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div dir="ltr">Devs, I have been trying to find the cause of this bug for some time, and I thought that someone might be able to provide a hint as to where to look to find the issue.<div>
<br></div><div>We have an external kernel that works very well when you connect onto it with a single frontend. But when you connect two frontends (console or notebook), then the conversation gets confused.</div>
<div><br></div><div>More detail:</div><div><br></div><div>Terminal 1:</div><div><br></div><div>ipython console --profile calico</div><div><br></div><div>works fine.</div><div><br></div><div>Terminal 2:</div><div><br></div>
<div><div>ipython console --profile calico --existing</div></div><div><br></div><div>starts, but now things are confused. Now, each terminal takes turns being able to get the prompt. If run in notebook, then code execution never signals the end of computing (the [*] never turns to a number, but you can still enter and get results back). In the console, output is received, but the prompt is in the other terminal.</div>
<div><br></div><div>It works fine with a single front end.</div><div><br></div><div>Any ideas appreciated!</div><span><font color="#888888"><div><br></div><div>-Doug</div></font></span></div>
<br></div></div>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div></div>