[IPython-dev] Parallel IPython: Spawn Tasks from Tasks, direct scheduling

[MinRK]

That's interesting.  If you are doing engine assignment yourself, I would
probably say that you should have that logic at a level slightly higher
than IPython.  Then on task submission, you can call out to a function
directly if your code determines that it should be run locally rather than
submitted via IPython.


On Thu, Oct 17, 2013 at 12:34 PM, Andreas Buhr <andreas at andreasbuhr.de>wrote:

> Hi MinRK,
>
> On 10/17/2013 09:20 PM, MinRK wrote:
> >
> >     And if I spawn a new task on a worker, and that new task is to be
> >     executed on the exact same machine, is there a way to execute that
> task
> >     without going back to the HUB/scheduler?
> >
> >
> > If you spawn new tasks, they are submitted in exactly the same way as
> > all other tasks via the scheduler. If you want it to run on the same
> > machine that's submitting the task, why not just call the function
> directly?
>
> Because I would like to have also task priorities and there might be
> tasks with higher priority waiting. Are there task priorities in IPython?
> Also because I would not like the task spawner to know where the task
> will be executed. At the end, I'd like to say "Call function A on object
> B with arguments C", and the system I am about to create should carry
> the task to B's home and execute it there. And it should not go over the
> wire if B happen to live on the same machine.
>
> Thanks again for your help :-)
>
> best regards,
> Andreas
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20131017/5c163911/attachment.html>