[IPython-dev] Two frontends, one kernel

Doug Blank doug.blank at gmail.com
Fri May 2 12:04:53 EDT 2014


On Thu, May 1, 2014 at 5:35 PM, MinRK <benjaminrk at gmail.com> wrote:

> ROUTER sockets use an IDENTITY prefix to determine who the recipient
> should be.
>

Ok, I think I understand, but still not working. Here is some wording that
(if correct) could be inserted in an expanded discussion of the wire
protocol (which doesn't yet mention socket type):
http://ipython.org/ipython-doc/dev/development/messaging.html#the-wire-protocol

"""
If you only want to implement basic functionality (one frontend per kernel,
no parallel or cluster support) then using DEALER instead of ROUTER is
fine, and a good place to start because you don't need the complexity of
the routing protocol. You can simply leave out the routing prefixes before
the <IDS|MSG> delimiter in the wire protocol.

If you do want more than the basic functionality (such as allowing two
frontends to a kernel) then you must use the ROUTER socket type, and must
have the correct routing prefixes.
"""

The problem is that I haven't gotten the routing to work, and it seems like
it deserves some documentation. For example, watching the IPython kernel, I
appear to see the following kinds of routing:

message type: "kernel_info_reply", routing appears to be "SESSION_ID":

*** KERNEL SENT: "['860e49d1-3852-4d9a-a623-01271cc95f21', '<IDS|MSG>',
'1832e7dad75f76550782d188b01d902c259da4b1d12ad9aa51c11bb756176065',
'{"date":"2014-05-02T11:36:54.231974","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"e4cc180a-d262-4ac8-b4e0-fbc9e49855fb","msg_type":"kernel_info_reply"}',
'{"date":"2014-05-02T11:36:53.956564","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"08428883-db50-401d-8601-e62fadcf38ba","msg_type":"kernel_info_request"}',
'{}',
'{"protocol_version":[4,1],"language_version":[2,7,5],"language":"python","ipython_version":[2,0,0,""]}']"

message type: "status", routing appears to be "kernel.KERNEL_ID.status":

*** KERNEL SENT: "['kernel.a5092211-ccf1-4017-bab9-58bab7d8eb52.status',
'<IDS|MSG>',
'1a3ac6b69c87a4e5c30ea95001c86868ee1af08b4e6d953f0478f117ac6f6124',
'{"date":"2014-05-02T11:37:56.826835","username":"kernel","session":"a97ba32b-b539-4b37-ab2e-08076f5a9bd6","msg_id":"fefb1c9c-8c93-4a7e-85a0-6440eda8af99","msg_type":"status"}',
'{"date":"2014-05-02T11:37:56.824306","username":"dblank","session":"860e49d1-3852-4d9a-a623-01271cc95f21","msg_id":"335597c6-2101-4343-8d64-2a9d328617ef","msg_type":"execute_request"}',
'{}', '{"execution_state":"busy"}']"

message type: "execute_repl", routing is "execute_reply", raw data (end of
message) is list of session ids:

*** KERNEL SENT: "[u'execute_reply', {'status': u'ok', 'execution_count':
2, 'user_variables': {}, 'payload': [], 'user_expressions': {}},
{'parent_header': {}, 'msg_type': 'execute_request', 'msg_id':
'4a153840-c5f1-4f24-813f-4585c9eb776f', 'content': {'code': '1 + 1\n',
'silent': False, 'user_variables': [], 'store_history': True,
'user_expressions': {}, 'allow_stdin': True}, 'header': {'date':
datetime.datetime(2014, 5, 2, 11, 42, 50, 892870), 'username': 'dblank',
'session': '1a402950-a0bf-417d-88d3-13b6d58506c3', 'msg_id':
'4a153840-c5f1-4f24-813f-4585c9eb776f', 'msg_type': 'execute_request'},
'buffers': [], 'metadata': {}}, {'dependencies_met': True, 'engine':
u'a5092211-ccf1-4017-bab9-58bab7d8eb52', 'status': u'ok', 'started':
datetime.datetime(2014, 5, 2, 11, 42, 50, 896000)},
['1a402950-a0bf-417d-88d3-13b6d58506c3']]"

Am I reading this correctly? If so, then many kernel developers may stop at
basic communication and use DEALER (which is fine, I guess). For those of
us you want to implement the full spec (with the expectation of having
parallel engines working), each of these cases will need to be dealt with,
right?

Thanks again!

-Doug



> In an IPython message, this would be the frames prior to the '<IDS|MSG>'
> delimiter. You must send these as the beginning of the reply in order for
> the reply to arrive at the correct peer.
>
> -MinRK
>
>
> On Thu, May 1, 2014 at 1:24 PM, Doug Blank <doug.blank at gmail.com> wrote:
>
>> On Thu, May 1, 2014 at 4:01 PM, MinRK <benjaminrk at gmail.com> wrote:
>>
>>> All of those DEALER sockets should be ROUTER sockets. The DEALER
>>> behavior is correct for only one peer, but it will round-robin replies if
>>> there is more than one.
>>>
>>
>> If I change the ShellChannel to ROUTER then the kernel will no longer
>> respond.
>>
>> If I change the others to ROUTER, then I don't see a difference with
>> prior behavior.
>>
>>
>>
>>>  Clearly some documentation is wrong. Where did you get the suggestion
>>> that they should be DEALER sockets?
>>>
>>
>> Probably a combination of finding what would work, and looking at other
>> code (it is sometimes hard to translate code and docs between different ZMQ
>> libraries).
>>
>> Thank you for helping me track this down!
>>
>> -Doug
>>
>>
>>>
>>> -MinRK
>>>
>>>
>>> On Thu, May 1, 2014 at 12:45 PM, Doug Blank <doug.blank at gmail.com>wrote:
>>>
>>>> On Thu, May 1, 2014 at 2:37 PM, MinRK <benjaminrk at gmail.com> wrote:
>>>>
>>>>> What are your socket types on the Kernel side?
>>>>>
>>>>
>>>> Here is the definitions for our channels:
>>>>
>>>>     public class ShellChannel : Channel {
>>>>     base(session, auth, transport, address, port, SocketType.DEALER) {
>>>>             }
>>>>     }
>>>> public class IOPubChannel : Channel {
>>>>     base(session, auth, transport, address, port, SocketType.PUB) {
>>>>             }
>>>>     }
>>>> public class ControlChannel : Channel {
>>>>     base(session, auth, transport, address, port, SocketType.DEALER) {
>>>>             }
>>>>     }
>>>> public class StdInChannel : Channel {
>>>>     base(session, auth, transport, address, port, SocketType.DEALER) {
>>>>             }
>>>>     }
>>>> public class HeartBeatChannel : Channel {
>>>>      base(session, auth, transport, address, port, SocketType.REP) {
>>>>             }
>>>>     }
>>>>
>>>> Could that really effect multi-frontend, but not single?
>>>>
>>>> -Doug
>>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Thu, May 1, 2014 at 9:58 AM, Doug Blank <doug.blank at gmail.com>wrote:
>>>>>
>>>>>> Devs, I have been trying to find the cause of this bug for some time,
>>>>>> and I thought that someone might be able to provide a hint as to where to
>>>>>> look to find the issue.
>>>>>>
>>>>>> We have an external kernel that works very well when you connect onto
>>>>>> it with a single frontend. But when you connect two frontends (console or
>>>>>> notebook), then the conversation gets confused.
>>>>>>
>>>>>> More detail:
>>>>>>
>>>>>> Terminal 1:
>>>>>>
>>>>>> ipython console --profile calico
>>>>>>
>>>>>> works fine.
>>>>>>
>>>>>> Terminal 2:
>>>>>>
>>>>>> ipython console --profile calico --existing
>>>>>>
>>>>>> starts, but now things are confused. Now, each terminal takes turns
>>>>>> being able to get the prompt. If run in notebook, then code execution never
>>>>>> signals the end of computing (the [*] never turns to a number, but you can
>>>>>> still enter and get results back). In the console, output is received, but
>>>>>> the prompt is in the other terminal.
>>>>>>
>>>>>> It works fine with a single front end.
>>>>>>
>>>>>> Any ideas appreciated!
>>>>>>
>>>>>> -Doug
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> IPython-dev mailing list
>>>>> IPython-dev at scipy.org
>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> IPython-dev mailing list
>>>> IPython-dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140502/73fa60b6/attachment.html>


More information about the IPython-dev mailing list