Traceback when using multiprocessing, less than helpful?
rosuav at gmail.com
Fri Nov 22 05:24:05 CET 2013
On Fri, Nov 22, 2013 at 2:57 PM, John Ladasky
<john_ladasky at sbcglobal.net> wrote:
> or, for that matter, why data needs to be pickled to pass it between processes.
Oh, that part's easy. Let's leave the multiprocessing module out of it
for the moment; imagine you spin up two completely separate instances
of Python. Create some object in one of them; now, transfer it to the
other. How are you going to do it?
Ultimately, the operating system isn't going to give you facilities
for moving complex objects around - what you almost exclusively get is
streams of bytes (or occasionally messaged chunks with lengths, but
still of bytes). Pickling is one method of turning an object into a
stream of bytes, in such a way that it can be turned back into an
equivalent object on the other side. And therein is the problem with
exceptions; since the traceback includes references to stack frames
and such, it's not as simple as saying "Two to beam up" and hearing
the classic sound effect - somehow you need to transfer all the
appropriate information across processes.
More information about the Python-list