[IPython-dev] IPython.parallel slow push

[Moritz Beber]

Dear all,

I often find myself in the situation where I use IPython.parallel to push a
bunch of data to the kernels (around 2 GB) and then do calculations over a
large parameter space on that data. This is convenient and simple enough to
do but copying the data takes a large amount of time (it scales with the
number of kernels).

Is this something to be avoided altogether, are there ways to speed it up?

I'd welcome any pointers.

Best,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20140811/70615b31/attachment.html>