[Python-Dev] Re: anonymous blocks

Guido van Rossum gvanrossum at gmail.com
Thu Apr 28 00:58:14 CEST 2005

> Probably my attempt at a *brief* explanation backfired.  No, they're not
> continuations or anything nearly that complicated.  I'm "just" simulating
> threads using generators that yield a nested generator when they need to do
> something that might block waiting for I/O.  The pseudothread object pushes
> the yielded generator-iterator and resumes it.  If that generator-iterator
> raises an error, the pseudothread catches it, pops the previous
> generator-iterator, and passes the error into it, traceback and all.
> The net result is that as long as you use a "yield expression" for any
> function/method call that might do blocking I/O, and those functions or
> methods are written as generators, you get the benefits of Twisted (async
> I/O without threading headaches) without having to "twist" your code into
> the callback-registration patterns of Twisted.  And, by passing in errors
> with tracebacks, the normal process of exception call-stack unwinding
> combined with pseudothread stack popping results in a traceback that looks
> just as if you had called the functions or methods normally, rather than
> via the pseudothreading mechanism.  Without that, you would only get the
> error context of 'async_readline()', because the traceback wouldn't be able
> to show who *called* async_readline.

OK, I sort of get it, at a very high-level, although I still feel this
is wildly out of my league.

I guess I should try it first. ;-)

> >In Python 3000 I want to make the traceback a standard attribute of
> >Exception instances; would that suffice?
> If you're planning to make 'raise' reraise it, such that 'raise exc' is
> equivalent to 'raise type(exc), exc, exc.traceback'.  Is that what you
> mean?  (i.e., just making it easier to pass the darn things around)
> If so, then I could probably do what I need as long as there exist no error
> types whose instances disallow setting a 'traceback' attribute on them
> after the fact.  Of course, if Exception provides a slot (or dictionary)
> for this, then it shouldn't be a problem.

Right, this would be a standard part of the Exception base class, just
like in Java.

> Of course, it seems to me that you also have the problem of adding to the
> traceback when the same error is reraised...

I think when it is re-raised, no traceback entry should be added; the
place that re-raises it should not show up in the traceback, only the
place that raised it in the first place. To me that's the essence of
re-raising (and I think that's how it works when you use raise without

> All in all it seems more complex than just allowing an exception and a
> traceback to be passed.

Making the traceback a standard attribute of the exception sounds
simpler; having to keep track of two separate arguments that are as
closely related as an exception and the corresponding traceback is
more complex IMO.

The only reason why it isn't done that way in current Python is that
it couldn't be done that way back when exceptions were strings.

> >I really don't want to pass
> >the whole (type, value, traceback) triple that currently represents an
> >exception through __next__().
> The point of passing it in is so that the traceback can be preserved
> without special action in the body of generators the exception is passing
> through.
> I could be wrong, but it seems to me you need this even for PEP 340, if
> you're going to support error management templates, and want tracebacks to
> include the line in the block where the error originated.  Just reraising
> the error inside the generator doesn't seem like it would be enough.

*** I have to think about this more... ***

> > > I think it'd be simpler just to have two methods, conceptually
> > > "resume(value=None)" and "error(value,tb=None)", whatever the actual method
> > > names are.
> >
> >Part of me likes this suggestion, but part of me worries that it
> >complicates the iterator API too much.
> I was thinking that maybe these would be a "coroutine API" or "generator
> API" instead.  That is, something not usable except with
> generator-iterators and with *new* objects written to conform to it.  I
> don't really see a lot of value in making template blocks work with
> existing iterators.

(You mean existing non-generator iterators, right? existing
*generators* will work just fine -- the exception will pass right
through them and that's exactly the right default semantics.

Existing non-generator iterators are indeed a different case, and this
is actually an argument for having a separate API: if the __error__()
method doesn't exist, the exception is just re-raised rather than
bothering the iterator.

OK, I think I'm sold.

> For that matter, I don't see a lot of value in
> hand-writing new objects with resume/error, instead of just using a generator.

Not a lot, but I expect that there may be a few, like an optimized
version of lock synchronization.

> So, I guess I'm thinking you'd have something like tp_block_resume and
> tp_block_error type slots, and generators' tp_iter_next would just be the
> same as tp_block_resume(None).
> But maybe this is the part you're thinking is complicated.  :)

No, this is where I feel right at home. ;-)

I hadn't thought much about the C-level slots yet, but this is a
reasonable proposal.

Time to update the PEP; I'm pretty much settled on these semantics now...

--Guido van Rossum (home page: http://www.python.org/~guido/)

More information about the Python-Dev mailing list