[IPython-dev] Parallel IPython: Spawn Tasks from Tasks, direct scheduling

MinRK benjaminrk at gmail.com
Thu Oct 17 15:20:31 EDT 2013


On Thu, Oct 17, 2013 at 12:16 PM, Andreas Buhr <andreas at andreasbuhr.de>wrote:

> Hi,
>
> wow, thanks a lot for your help, that's great.
>
> On 10/17/2013 09:06 PM, MinRK wrote:
> > A notebook illustrating both approaches
> > <http://nbviewer.ipython.org/7030408>
>
> >From inside of those member functions, can I spawn new tasks? Can I just
> get a new 'Client()', even when I am on a worker?
>

You can create a client and submit new tasks from there,
though there can be hazards associated with that in terms of the number of
sockets created.


>
> And if I spawn a new task on a worker, and that new task is to be
> executed on the exact same machine, is there a way to execute that task
> without going back to the HUB/scheduler?
>

If you spawn new tasks, they are submitted in exactly the same way as all
other tasks via the scheduler. If you want it to run on the same machine
that's submitting the task, why not just call the function directly?

-MinRK


>
> thanks a lot again,
> best regards,
> Andreas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20131017/7512d027/attachment.html>


More information about the IPython-dev mailing list