Mark Hammond wrote:
While I concede that this is likely to work in the general case, I am not sure it is "correct". If no threading semantics will be broken by having one thread use multiple thread-states, then I must ask what purpose thread-states (as opposed to the GIL) have.
That is easy to answer (even though it is out of my area of expertise): it carries the Python stack, in particular for exceptions.
It also carries the profiler and debugger hooks, a general purpose "thread state" dictionary and other misc. details such as the tick count, recursion depth protection etc.
Now, if you have multiple thread states in a single thread, the question is how a Python exception should propagate through the C stack.
Actually, I think the question is still "why would a single thread have multiple thread-states?". (Or maybe "should a thread-state be renamed to, say, an "invocation frame"?)
With multiple thread states, the exception "drops off" in the callback, which usually
"usually" is the key word here. Python isn't designed only to handle what programs "usually" do. A strategy I have seen recently here, which is to argue that any other requirements are self-evidently broken, is not helpful. We could possibly argue that exceptions are OK to handle this way. Similar amounts of text could also possibly convince that the profiler, debugger and thread-switch items also will not be too badly broken by having multiple thread states per thread, or that such breakage is "desirable" (ie, can be justified). You will have more trouble convincing me that future items stored in a Python thread state will not be broken, but I am past arguing about it. Please correct me if I am wrong, but it seems your solution to this is: * Every function which *may* trigger such callbacks *must* switch out the current thread state (thereby dropping the GIL etc) * Every entry-point which needs to call Python must *always* allocate and switch to a new thread-state. * Anything broken by having multiple thread-states per thread be either (a) fixed, or (b) justified in terms of a specific CORBA binding implementation. * Anyone wanting anything more remains out on their own, just as now. If so, I am afraid I was hoping for just a little more <wink>. Mark.