[IPython-dev] Kernels and parallel support?

Doug Blank doug.blank at gmail.com
Fri Mar 21 08:36:03 EDT 2014


On Thu, Mar 20, 2014 at 4:06 PM, Doug Blank <doug.blank at gmail.com> wrote:
> On Thu, Mar 20, 2014 at 2:19 PM, MinRK <benjaminrk at gmail.com> wrote:
>>
>>
>>
>> On Thu, Mar 20, 2014 at 10:58 AM, Doug Blank <doug.blank at gmail.com> wrote:
>>>
>>> After reading:
>>>
>>> http://ipython.org/ipython-doc/dev/parallel/parallel_intro.html
>>>
>>> and related pages, I am unsure what the relationship between parallel
>>> processing and kernels are. Can external kernels be used in a parallel
>>> environment? Or is this part of IPython python-kernel specific? Does
>>> it use ZMQ messages?
>>
>>
>> Depends what you mean by 'external'. You can start engines at any time on
>> any machine, and they can join a cluster.  Yes, IPython.parallel uses zmq.
>
> If I wanted to explore using a cluster of 3rd-party kernels, such as
> IHaskell or ICalico, in a parallel environment, where would I begin?
> Has anyone done this before?
>
> I see here:
>
> http://ipython.org/ipython-doc/dev/parallel/magics.html#engines-as-kernels
>
> that kernels connect to a controller to create an engine. If the
> standard ipcontrollers can start up external kernels, I guess a place
> to begin might be implementing what the %px magic does?

Along with those general questions, a specific one:

I would have thought that "ipcluster start --profile calico" would
have used my c.KernelManager.kernel_cmd to start my kernels, but I
can't get ipcluster to use it. Likewise, if I use the "Cluster" tab in
the notebook to start a cluster, I also don't get my kernel.

How to start a cluster of third-party kernels?

-Doug

> -Doug
>
>> -MinRK
>>
>>>
>>>
>>> Thanks for any additional data,
>>>
>>> -Doug
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>



More information about the IPython-dev mailing list