<br><br><div class="gmail_quote">On Fri, Sep 7, 2012 at 2:18 PM, Michael Bilow <span dir="ltr"><<a href="mailto:michael.k.bilow@gmail.com" target="_blank">michael.k.bilow@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Wow, you nailed it, thanks. I was running ipcluster start in a different directory from the one my code was in. I still don't understand the provenance of that particular error, but I can see how it could cause one.</blockquote>
<div><br></div><div>Here's a mor detailed exploration of what is happening:</div><div><br></div><div> import pickle</div><div> from wrapper import parallel_run</div><div> print pickle.dumps(parallel_run)</div>
<div><br></div><div>gives:</div><div><br></div><div><div> cwrapper</div><div> parallel_run</div><div> p0</div><div> .</div></div><div><br></div><div>In other words, the serialized form of a module function is just the function name and the module in which it lives.</div>
<div><br></div><div>When the request arrives on the engines, it is deserialized back into its module namespace <a href="https://github.com/ipython/ipython/blob/master/IPython/utils/pickleutil.py#L98">like so</a>:</div><div>
<div><br></div><div> try:</div><div> __import__(self.module)</div><div> except ImportError:</div><div> pass</div><div> else:</div><div> g = sys.modules[self.module].__dict__</div>
</div><div><br></div><div>So what is happening is the import of `wrapper` is failing, but IPython suppressed that error, and unpacked the function into an empty namespace instead. It's possible that this error should *not* be suppressed, because it certainly would have been more informative to you to see `ImportError: wrapper` rather than a NameError since it came out in the wrong place.</div>
<div><br></div><div>I don't know what case is actually helped by this suppression, so I am inclined to just <a href="https://github.com/ipython/ipython/pull/2395">cease catching the error</a>.</div><div><br></div><div>
-MinRK</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<br></div><div>Thanks!</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Michael</div></font></span><div class="HOEnZb"><div class="h5"><div><br><div class="gmail_quote">On Fri, Sep 7, 2012 at 2:04 PM, MinRK <span dir="ltr"><<a href="mailto:benjaminrk@gmail.com" target="_blank">benjaminrk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><br><div class="gmail_quote"><div><div>On Fri, Sep 7, 2012 at 1:48 PM, Michael Bilow <span dir="ltr"><<a href="mailto:michael.k.bilow@gmail.com" target="_blank">michael.k.bilow@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi!<div><br></div><div>I'm new to this list and to IPython, and I started using IPython to parallelize some code I'd written (complete code: <a href="https://github.com/chuyelchulo/ipython-nameerror" target="_blank">https://github.com/chuyelchulo/ipython-nameerror</a>, requires numpy and PIL). I've been encountering a confusing NameError that keeps popping up when I try to run this program: The function to be mapped (parallel_run) is imported from a module different from __main__, but the interpreter complains that the functions and modules used by parallel_run are not part of the namespace.</div>
<div><br></div><div>I asked about the error here (<a href="http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues" target="_blank">http://stackoverflow.com/questions/12304847/ipython-parallel-computing-namespace-issues</a>), and was told to come to this list.</div>
</blockquote><div><br></div></div></div><div>Hi again!</div><div><br></div><div>When I do the following:</div><div><br></div><div>$> cd /path/to/ipython-nameerror</div><div>$> ipcluster start -n 4 --daemon</div><div>
$> python wrapper_wrapper.py</div>
<div>Time elapsed: 5.85908985138 s</div><div><br></div><div>It seems to work as expected.</div><div><br></div><div>A few quick questions:</div><div><br></div><div>1. what is the cwd when you run ipcluster? Are your engines local or remote? What do you get from:</div>
<div><br></div><div><div>import os</div><div>import pprint</div><div>from IPython import parallel</div><div><br></div><div>client = parallel.Client()</div><div>pprint.pprint(client[:].apply_async(os.getcwdu).get_dict())</div>
</div><div><br></div><div>or:</div><div><br></div><div>%px import wrapper</div><div><br></div><div>If I start the engines somewhere else such that `wrapper` is not importable, I get the NameErrors you are seeing.</div><div>
<br></div><div>-MinRK</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br></div><div>Thanks!</div><span><font color="#888888"><div><br></div><div>Michael</div><div>
</div>
</font></span><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br>
<br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org" target="_blank">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
IPython-dev mailing list<br>
<a href="mailto:IPython-dev@scipy.org">IPython-dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/ipython-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/ipython-dev</a><br>
<br></blockquote></div><br>