[Python-ideas] Protecting finally clauses of interruptions

[Yury Selivanov]

On 2012-04-04, at 4:43 PM, Paul Colomiets wrote:
>> How does it work?  To what object are you actually attaching timeout?
>> 
> 
> There is basically a "Coroutine" object. It's actually a list with
> paused generators, with top of them being currently running
> (or stopped for doing IO). It represents stack, because there
> is no built-in stack for generators.

Interesting.  I decided to go with a simple coroutine objects with
a '_caller' pointer to maintain the stack virtually.

> This framework does timeout handling in described way:
> 
> https://github.com/tailhook/zorro
> 
> Although, it's using greenlets. The difference is that we
> we don't need to keep a stack in our own scheduler
> when using greenlets, but everything else applies.

Are you using that particular framework (zorro)?  Or some modification
of it that uses generators too?

>> Sorry, I must had it explained in more details.  Right now we
>> interrupt code only where we have a 'yield', a greenlet.switch(),
>> or at the end of finally block, not at some arbitrary opcode.
>> 
> 
> Sure I do similar. But it doesn't work with threads, as
> they have no explicit yield or switch.

OK.

> On Wed, Apr 4, 2012 at 11:07 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>> On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:
>> 
>>> But anyway I don't see a reason to protect a single frame,
>>> because even if you have a simple mutex without coroutines
>>> you end up with:
>> 
>> BTW, for instance, in our framework each coroutine is a special
>> object that wraps generator/plain function.  It controls everything
>> that the underlying generator/function yields/returns.  But the actual
>> execution, propagation of returned values and raised errors is the
>> scheduler's job.  So when you are yielding a coroutine from another
>> coroutine, frames are not even connected to each other, since the
>> actual execution of the callee will be performed by the scheduler.
>> It's not like a regular python call.
>> 
> 
> Same applies here. But you propagate return value/error right?
> So you can't say "frames are not connected". They aren't from
> the interpreter point of view. But they are logically connected.

OK, we're on the same page.  '''"frames are not connected" from the
interpreter point of view''', that essentially means that 'f_in_finally'
will always be a flag related only to its own frame, right?  Any 
'propagation' of this flag is the responsibility of framework developers.

> So for example:
> 
> def a():
>  yield b()
> 
> def b():
>  yield
> 
> If `a().with_timeout(0.1)` is interrupted when it's waiting for value
> of `b()`, will `b()` continue it's execution?

Well, in our framework, if a() is getting aborted after it's scheduled
b(), but before it received the result of b(), we interrupt both of them
(and those that b() might had called).

>> I think even if it's decided to implement just your proposal, I feel
>> that 'f_in_finally' should indicate the state of only its *own* frame.
> 
> That was original intention. But it requires stack traversing. Andrew
> proposed to propagate this flag, which is another point of view
> on the same thing (not sure which one to pick though)

Again, if coroutines' frames aren't connected on the interpreter level
(it's the responsibility of a framework), about what exact propagation
are you and Andrew talking (in the sole context of the patch to cpython)?

-
Yury