[IPython-dev] IPython.parallel slow push

Moritz Beber moritz.beber at gmail.com
Mon Aug 11 08:52:13 EDT 2014

Dear all,

I often find myself in the situation where I use IPython.parallel to push a
bunch of data to the kernels (around 2 GB) and then do calculations over a
large parameter space on that data. This is convenient and simple enough to
do but copying the data takes a large amount of time (it scales with the
number of kernels).

Is this something to be avoided altogether, are there ways to speed it up?

I'd welcome any pointers.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/70615b31/attachment.html>

More information about the IPython-dev mailing list