[IPython-dev] Confusing NameError in Parallelizing with IPython
benjaminrk at gmail.com
Fri Sep 7 19:55:26 EDT 2012
On Fri, Sep 7, 2012 at 2:18 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:
> Wow, you nailed it, thanks. I was running ipcluster start in a different
> directory from the one my code was in. I still don't understand the
> provenance of that particular error, but I can see how it could cause one.
Here's a mor detailed exploration of what is happening:
from wrapper import parallel_run
In other words, the serialized form of a module function is just the
function name and the module in which it lives.
When the request arrives on the engines, it is deserialized back into its
module namespace like
g = sys.modules[self.module].__dict__
So what is happening is the import of `wrapper` is failing, but IPython
suppressed that error, and unpacked the function into an empty namespace
instead. It's possible that this error should *not* be suppressed, because
it certainly would have been more informative to you to see `ImportError:
wrapper` rather than a NameError since it came out in the wrong place.
I don't know what case is actually helped by this suppression, so I am
inclined to just cease catching the
> On Fri, Sep 7, 2012 at 2:04 PM, MinRK <benjaminrk at gmail.com> wrote:
>> On Fri, Sep 7, 2012 at 1:48 PM, Michael Bilow <michael.k.bilow at gmail.com>wrote:
>>> I'm new to this list and to IPython, and I started using IPython to
>>> parallelize some code I'd written (complete code:
>>> https://github.com/chuyelchulo/ipython-nameerror, requires numpy and
>>> PIL). I've been encountering a confusing NameError that keeps popping up
>>> when I try to run this program: The function to be mapped (parallel_run) is
>>> imported from a module different from __main__, but the interpreter
>>> complains that the functions and modules used by parallel_run are not part
>>> of the namespace.
>>> I asked about the error here (
>>> and was told to come to this list.
>> Hi again!
>> When I do the following:
>> $> cd /path/to/ipython-nameerror
>> $> ipcluster start -n 4 --daemon
>> $> python wrapper_wrapper.py
>> Time elapsed: 5.85908985138 s
>> It seems to work as expected.
>> A few quick questions:
>> 1. what is the cwd when you run ipcluster? Are your engines local or
>> remote? What do you get from:
>> import os
>> import pprint
>> from IPython import parallel
>> client = parallel.Client()
>> %px import wrapper
>> If I start the engines somewhere else such that `wrapper` is not
>> importable, I get the NameErrors you are seeing.
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>> IPython-dev mailing list
>> IPython-dev at scipy.org
> IPython-dev mailing list
> IPython-dev at scipy.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the IPython-dev