[IPython-dev] IPython parallel and objects
Moritz Beber
moritz.beber at gmail.com
Wed Jan 28 10:18:29 EST 2015
Hi,
On Wed, Jan 28, 2015 at 1:40 PM, thwiouz <maidos93 at laposte.net> wrote:
> Hi guys,
>
> I'd like to parallelize the following loop:
>
> m1 = np.arange(0, 10, 100)
> m2 = np.arange(0, 10, 100)
>
> for m_ in m1:
> for n_2 in m2:
> tmp = optimizer.compute_expectation([m_, n_])
> res.append(tmp)
>
> with type(optimizer) some class that I've defined. How could I handle it?
>
> I tried to do the same as in
>
> http://stackoverflow.com/questions/9363118/parallelise-nested-for-loop-in-ipython
> so I created a lambda function f
>
> f = lambda x, y: optimizer.compute_expectation([x, y])
>
> but it's not working with a load balanced view, saying that optimizer is
> not
> defined.
>
IPython only automatically transmits the function definition used in a
`map` call or similar. Your optimizer is inside your lambda function and
thus unknown on the remote kernels. In order to overcome this, take a look
here:
http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#remote-function-decorators
and here:
http://ipython.org/ipython-doc/2/parallel/parallel_task.html#dependencies
this could be helpful, too:
http://ipython.org/ipython-doc/2/parallel/parallel_multiengine.html#moving-python-objects-around
HTH,
Moritz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20150128/91860336/attachment.html>
More information about the IPython-dev
mailing list