Traceback when using multiprocessing, less than helpful?
john_ladasky at sbcglobal.net
Thu Nov 21 22:49:53 CET 2013
I didn't need to go as far as Chris Angelico's second suggestion. I haven't looked at certain parts of my own code for a while, but it turns out that I wrote it REASONABLY logically...
My evaluate() calls another function through pool.map_async() -- _evaluate(), which actually processes the data, on a single CPU. So I didn't need to hassle with threading, as Chris suggested. All I did was to import _evaluate in my top-level code, then change my function calls from evaluate() to _evaluate(). Out popped my numpy error, with a proper traceback. I can now debug it!
I can probably refactor my code to make it even cleaner. I'll have to deal with the fact that pool.map() requires that all arguments to each subprocess be submitted as a single, iterable object. I didn't want to have to do this when I only had a single process to run, but perhaps the tradeoff will be acceptable.
So now, for anyone who is still reading this: is it your opinion that the traceback that I obtained through multiprocessing.pool._map_async().get() SHOULD have allowed me to see what the ultimate cause of the exception was? I think so. Is it a bug? Should I request a bugfix? How do I go about doing that?
More information about the Python-list