[IPython-dev] running shell command from engine
Antonio González Peña
antgonza at gmail.com
Thu Jun 19 14:42:28 EDT 2014
Thanks for the reply.
I'm not really being able to use it, I tried:
from IPython.parallel import Client
rc = Client()
with rc[:].sync_imports():
from subprocess import Popen
v = rc.load_balanced_view()
result = v.map(lambda job: Popen([job]), jobs)
print list(result)
but get:
ValueError: Sorry, cannot pickle functions with closures
Then, I tried using dill as described here:
http://nbviewer.ipython.org/gist/minrk/5241793, so:
from types import FunctionType
import dill as pickle
from IPython.utils.pickleutil import can_map
from IPython.kernel.zmq import serialize
from IPython.parallel import Client
can_map.pop(FunctionType, None)
serialize.pickle = pickle
rc = Client()n
with rc[:].sync_imports():
from subprocess import Popen
v = rc.load_balanced_view()
result = v.map(lambda job: Popen([job]), jobs)
print list(result)
but now it fails with:
raise self._exception
IPython.parallel.error.CompositeError: one or more exceptions from
call to method: remote_import
[0:apply]: AttributeError: 'module' object has no attribute '__main__'
[1:apply]: AttributeError: 'module' object has no attribute '__main__'
[2:apply]: AttributeError: 'module' object has no attribute '__main__'
[3:apply]: AttributeError: 'module' object has no attribute '__main__'
Any ideas?
On Thu, Jun 19, 2014 at 12:29 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Sure! Popen won't give you an actual result that's very useful, you'll need
> to decide whether to capture stdout/err, to use pipes, etc. But
> subprocess/popen are just python functions like any other, so you can use
> them all the same...
>
> f
>
>
> On Thu, Jun 19, 2014 at 6:59 AM, Antonio González Peña <antgonza at gmail.com>
> wrote:
>>
>> Hello,
>>
>> Is it possible to run a shell command from an engine? For example:
>> result = v.map(lambda job: Popen(['/bin/sh', job]), jobs)
>>
>> Thanks
>>
>> --
>> Antonio
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
>
>
> --
> Fernando Perez (@fperez_org; http://fperez.org)
> fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
> fernando.perez-at-berkeley: contact me here for any direct mail
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
--
Antonio
More information about the IPython-dev
mailing list