<div dir="ltr">Ah, that's just a limitation of the fact that your lambda is capturing the enclosing scope.<div><br></div><div>Just define a normal function with local imports, and you're set.</div><div><br></div><div>
def mytool():</div><div> import suprocess as sp</div><div> sp.popen(...)</div><div><br></div><div>v.map(mytool, ....)</div><div><br></div><div>Cheers</div><div><br></div><div>f</div></div><div class="gmail_extra"><br><br>
<div class="gmail_quote">On Thu, Jun 19, 2014 at 11:42 AM, Antonio González Peña <span dir="ltr"><<a href="mailto:antgonza@gmail.com" target="_blank">antgonza@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks for the reply.<br>
<br>
I'm not really being able to use it, I tried:<br>
from IPython.parallel import Client<br>
<br>
rc = Client()<br>
with rc[:].sync_imports():<br>
from subprocess import Popen<br>
<br>
v = rc.load_balanced_view()<br>
<br>
result = v.map(lambda job: Popen([job]), jobs)<br>
print list(result)<br>
but get:<br>
ValueError: Sorry, cannot pickle functions with closures<br>
<br>
Then, I tried using dill as described here:<br>
<a href="http://nbviewer.ipython.org/gist/minrk/5241793" target="_blank">http://nbviewer.ipython.org/gist/minrk/5241793</a>, so:<br>
from types import FunctionType<br>
import dill as pickle<br>
from IPython.utils.pickleutil import can_map<br>
from IPython.kernel.zmq import serialize<br>
from IPython.parallel import Client<br>
<br>
can_map.pop(FunctionType, None)<br>
serialize.pickle = pickle<br>
<br>
rc = Client()n<br>
with rc[:].sync_imports():<br>
from subprocess import Popen<br>
<br>
v = rc.load_balanced_view()<br>
<br>
result = v.map(lambda job: Popen([job]), jobs)<br>
print list(result)<br>
but now it fails with:<br>
raise self._exception<br>
IPython.parallel.error.CompositeError: one or more exceptions from<br>
call to method: remote_import<br>
[0:apply]: AttributeError: 'module' object has no attribute '__main__'<br>
[1:apply]: AttributeError: 'module' object has no attribute '__main__'<br>
[2:apply]: AttributeError: 'module' object has no attribute '__main__'<br>
[3:apply]: AttributeError: 'module' object has no attribute '__main__'<br>
<br>
Any ideas?<br>
<div class="HOEnZb"><div class="h5"><br>
On Thu, Jun 19, 2014 at 12:29 PM, Fernando Perez <<a href="mailto:fperez.net@gmail.com">fperez.net@gmail.com</a>> wrote:<br>
> Sure! Popen won't give you an actual result that's very useful, you'll need<br>
> to decide whether to capture stdout/err, to use pipes, etc. But<br>
> subprocess/popen are just python functions like any other, so you can use<br>
> them all the same...<br>
><br>
> f<br>
><br>
><br>
> On Thu, Jun 19, 2014 at 6:59 AM, Antonio González Peña <<a href="mailto:antgonza@gmail.com">antgonza@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Hello,<br>
>><br>
>> Is it possible to run a shell command from an engine? For example:<br>
>> result = v.map(lambda job: Popen(['/bin/sh', job]), jobs)<br>
>><br>
>> Thanks<br>
>><br>
>> --<br>
>> Antonio<br>
>> _______________________________________________<br>
>> IPython-dev mailing list<br>
>> <a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
>> <a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
><br>
><br>
><br>
><br>
> --<br>
> Fernando Perez (@fperez_org; <a href="http://fperez.org" target="_blank">http://fperez.org</a>)<br>
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)<br>
> fernando.perez-at-berkeley: contact me here for any direct mail<br>
><br>
> _______________________________________________<br>
> IPython-dev mailing list<br>
> <a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
> <a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
><br>
<br>
<br>
<br>
--<br>
Antonio<br>
_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Fernando Perez (@fperez_org; <a href="http://fperez.org" target="_blank">http://fperez.org</a>)<br>fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)<br>
fernando.perez-at-berkeley: contact me here for any direct mail<br>
</div>