On Tue, Nov 1, 2011 at 4:27 PM, Terry Reedy firstname.lastname@example.org wrote:
I believe raise just instantiates the indicated exception. I expect that Exception.__new__ or .__init__ captures the traceback info. Subclasses can add more. A SuspendExecution exception should be able to grab as much as is needed for a resume. A CAPI call could be added if needed.
No, the traceback info is added by the eval loop itself. Remember that when you raise an exception *type* (rather than an instance), the exception doesn't get instantiated until it gets caught somewhere - the eval loop maintains the unwinding stack for the traceback as part of the thread state until it is time to attach it to the exception object.
This is all at the tail end of the eval loop in CPython, but be warned it's fairly brain bending stuff that depends on various internal details of the eval loop: http://hg.python.org/cpython/file/default/Python/ceval.c#l2879
I hope you keep looking at this idea. Function calls stop execution and pass control 'down', to be resumed by return. yield stops execution and passes control 'up', to be resumed by next (or .send). Exceptions pass control 'up' (or 'out') without the possibility of resuming. All that is lacking is something to suspend and pass control 'sideways', to a specific target. A special exception makes some sense in that exceptions already get the call stack needed to resume after suspension.
That's not actually true - due to the need to process exception handling clauses and finally blocks (including the implicit ones inside with statements), the internal state of those frames is potentially no longer valid for resumption (they've moved on beyond the point where the internal function was called).
I'll also note that it isn't necessary to pass control sideways, since there are two different flavours of coroutine design (the PDF article in the other thread describes this well). The Lua version is "asymmetric coroutines", and they only allow you to return to the point that first invoked the coroutine (this model is a fairly close fit with Python's generators and exception handling). The greenlet version is "symmetric" coroutines, and those let you switch directly to any other coroutine.
Both models have their pros and cons, but the main advantage of asymmetric coroutines is that you can just say "suspend this thread" without having to say *where* you want to switch to. Of course, you can implement much the same API with symmetric coroutines as well, so long as you can look up your parent coroutine easily. Ultimately, I expect the symmetric vs asymmetric decision will be driven more by implementation details than by philosophical preferences one way or the other.
I will note that Ron's suggestion to leverage the existing eval loop stack collection provided by the exception handling machinery does heavily favour the asymmetric approach. Having a quick look to refresh my memory of some of the details of CPython's exception handling, I've come to the following tentative conclusions:
- an ordinary exception won't do, since you don't want to trigger except and finally blocks in outer frames (ceval.c#2903) - in CPython, a new "why = WHY_SUSPEND" at the eval loop layer is likely a better approach, since it would allow the frame stack to be collected without triggering exception handling - the stack unwinding would then end when a "SETUP_COCALL" block was encountered on the block stack (just as SETUP_EXCEPT and SETUP_FINALLY can stop the stack unwinding following an exception - with the block stacks within the individual frames preserved, the collected stack should be in a fit state for later restoration - the "fast_yield" code and the generator resumption code should also provide useful insight
There's nothing too magical there - once we disclaim the ability to suspend coroutines while inside a C function (even one that has called back in via the C/Python API), it should boil down to a combination of the existing mechanics for generators and exception handling. So, even though the above description is (highly) CPython specific, it should be feasible for other implementations to come up with something similar (although perhaps not easy: http://lua-users.org/lists/lua-l/2007-07/msg00002.html).