[Python-Dev] Asynchronous use of Traceback objects

[Christopher Armstrong]

With the implementation and soon release of PEP 342, I've been
thinking more about traceback objects lately. First I'll give you some
background information for my problem.

I've implemented a module for integrating Twisted's Deferreds with the
new yield expression, that allows you to do stuff like:

@defgen
def foo():
    x = yield deferredOperation()
    print x

Basically, defgen checks for Deferreds coming out of the generator,
and when it finds one, adds a callback to that Deferred which will
resume the generator, sending the result in. Since Deferreds have
error support, it also allows code like this:

@defgen
def foo(userinput):
    try: 
        x = yield deferredOperation(userinput)
    except ValueError: 
        "Crap, wrong user input!"

We have this object in Twisted called the "Failure", which is used for
conveniently passing around exceptions asynchronously, and Deferreds
use them to represent errors in deferred operations. The Failure
objects have a reference to an exception object and the traceback that
was associated with the original raising of that exception.  However,
we can only hold on to that traceback for so long, because traceback
objects have references to so many things and can so easily cause
horrible GC issues. The loss of this traceback object isn't *usually*
a problem because we store enough other information in the Failure
object to print representations of tracebacks nicely. However, if we
try to re-raise the exception, we lose that traceback information.

defgen re-raises the exception from a Failure into a defgen-using
function with g.throw(). Unfortunately, quite often the traceback has
been cleaned from the Failure object, and this means you'll get
exceptions in defgen-using code with very bizarre and uninformative
tracebacks.

I had the idea to create a fake Traceback object in Python that
doesn't hold references to any frame objects, but is still able to be
passed to 'raise' and formatted as tracebacks are, etc. Unfortunately,
raise does a type check on its third argument and, besides, it seems
the traceback formatting functions are very reliant on the internal
structure of traceback objects, so that didn't work.

It does seem that I would be able to construct a passable fake
Traceback object from C code -- one that had references to fake
frames. These fake objects would only remember the original line
numbers, filenames and so forth so that traceback printing could still
work. I can try implementing this soon, but I'd just like to make sure
I'm on the right track. For example, perhaps a better idea would be to
change the traceback-printing functions to use Python attribute lookup
instead of internal structure lookup, and then change raise to accept
arbitrary Python objects as its third argument, as long as it matches
the traceback interface. That would probably mean much more work,
though.

One concern is that I really don't like requiring C modules to use
Twisted; all of the ones currently in there are optional. What's the
likelihood of such a traceback-constructor getting its way into
CPython if I do implement it? It may seem niche, but I expect many
Twisted users would like to use PEP 342 defgen (many users are already
using the defgen I wrote for python 2.2 generators).

Thanks for any help, and have fun,
-- 
  Twisted   |  Christopher Armstrong: International Man of Twistery
   Radix    |    -- http://radix.twistedmatrix.com
            |  Release Manager, Twisted Project
  \\\V///   |    -- http://twistedmatrix.com
   |o O|    |  
w----v----w-+