Appending traceback from exception in child thread

[Chris Mellon]

On Fri, May 15, 2009 at 1:09 PM, Edd <edd at nunswithguns.net> wrote:
> Hi folks,
>
> I have a some threadpool code that works like this :
>
>    tp = ThreadPool(number_of_threads)
>    futures = [tp.future(t) for t in tasks] # each task is callable
>    for f in futures:
>        print f.value() # <-- may propagate an exception
>
> The idea being that a Future object represents a value that may or may
> not have been computed yet and the .value() method will block until
> that value is ready.
>
> As the comment on the last line indicates, looking at the .value() of
> a Future may give the return value of the associated task, or it may
> also propagate an exception that was raised in a child thread.
>
> Inside the implementation I store caught exceptions with code that
> looks something like:
>
>    try:
>        self.return_value = self.task()
>    except:
>        self.exception = sys.exc_info()[1]
>
> The problem I have is that when an exception is propagated in to the
> parent thread by re-raising it, the part of the traceback from the
> child thread is lost. So if the program dies due to such an exception,
> I can't see what caused it by examining the printed traceback.
>
> To solve this problem, I could of course grab the traceback in the
> child thread and create some kind of custom exception that stashes the
> trace inside. This could then be caught on the fringes of my code in
> order to combine the tracebacks of parent and child together before
> printing it. But I was wondering if there might be a nicer way that
> keeps the original exception object. Perhaps something akin to gluing
> the tracebacks together at the point where the exception is re-raised?
>
> Kind regards,
>
> Edd
> --
> http://mail.python.org/mailman/listinfo/python-list
>


Take a look at the Deferred and Failure objects from Twisted. Deferred
is a deferred result with callback (basically a Future without the
blocking interface), and Failure is an asynchronous exception object.