On 2012-04-02, at 6:33 PM, Paul Colomiets wrote:

...

How's that going to work for tons of existing code?

It isn't. But it doesn't break code any more than it already is. Your proposal doesn't solve any problems with existing code too.

But anyway I don't propose any new ways to interrupt code I only propose a way to inform trampoline when it's unsafe to interrupt code.

Well, if we're thinking only about interrupting coroutines (not threads), then it's going to work, yes. My initial desire to use a special exception for the purpose, was because of: - it's easier to throw exception in the thread (the C-API function already exists, and we need to think about consequences of using it) - PyPy disables the JIT when working with frames (if I recall correctly). That's why I wanted 'f_in_finally' to be an implementation detail of CPython, hidden from the user code. Perhaps PyPy could implement the handling of our special exception in a more efficient way, without the side-effect of disabling the JIT. What do you think? - Yury

On 2012-04-02, at 7:36 PM, Paul Colomiets wrote:

It's nice for python to have finally protection built-in, but I don't see how it can be implemented in a generic way.

How about adding some sort of 'interruption protocol'? Say Threads, generators, and Greenlets will have a special method called '_interrupt'. While it will be implemented differently for each of them, it will work on the same principle: - check if the underlying code block is in one of its 'finally' statements - if it is: set a special flag to abort when the frame's counter of 'finally' blocks reaches 0 to raise the ExecutionInterrupt exception - if it is not: raise the ExecutionInterrupt exception right away - Yury

On 2012-04-03, at 3:16 AM, Paul Colomiets wrote:

1. For yield-based coroutines you must inspect stack anyway, since interpreter doesn't have a stack, you build it yourself (although, I don't know how `yield from` changes that)

2. For greenlet based coroutines it is unclear what the stack is. For example:

def f1(): try: pass finally: g1.switch()

def f2(): sleep(1.0)

g1 = greenlet(f1) g2 = greenlet(f2) g1.switch()

Is it safe to interrupt g2 while it's in `sleep`? (If you wonder how I fix this problem with f_in_finally stack, it's easy. I usually switch to a coroutine from trampoline, so this is a boundary of the stack which should be checked for f_in_finally).

Wait. So you're tracing the whole coroutine execution stack to check if the current coroutine was called in a finally block of some other coroutine? For handling timeouts I don't think that is necessary (maybe there are other use cases?) In the example below you actually have to interrupt g2: def g1(): try: ... finally: g2().with_timeout(0.1) def g2(): sleep(2) You shouldn't guarantee that the *whole* chain of functions/ coroutines/etc will be safe in their finally statements, you just need to protect the top coroutines in the timeouts queue. Hence, in the above example, if you run g1() with a timeout, the trampoline should ensure that it won't interrupt it while it is in its finally block. But it can interrupt g2() in any context at any point of its execution. And if g2() gets interrupted, g1()'s finally statement will be broken, yes. But that's the responsibility of the developer to ensure that the code in 'finally' handles exceptions within it correctly. That's just my approach to handle timeouts, I'm not advocating it to be the very right one. Are there any other use-cases when you have to inspect the execution stack? Because if there is no, 'interrupt()' method is sufficient and implementable, as both generators and greenlets are well aware about the code frames they holding.

3. For threads it was discussed several times and rejected. This proposal may make thread interruptions slightly safer, but I'm not sure it's enough to convince people.

That's why I'm advocating for a PEP. Thread interruption isn't a safe feature in the .NET CLR either. You may break things with it there too. And it doesn't protect the chain of functions calling each other from their 'finally' statements, it just protects the top frame. The 'abort' and 'interrupt' methods aren't advertised to be used in .NET, use them at your own risk. So I don't think that we can, or should ensure 100% safety when interrupting a thread. And that's why I think it is worth to propose a mechanism that will work for many concurrency primitives.

So I still propose add a frame flag, which doesn't break anything, and gives us experiment with interruptions without putting some experimental code into the core.

There are cons and pros in your solution. Pros ---- - can be used right away in coroutine libraries. - somewhat simple and small CPython patch. Cons ---- - you have to work with frames almost throughout the execution of the program. In PyPy you simply will have the JIT disabled. And I'm not sure how frame access works in Jython and IronPython from the performance point of view. - no mechanism for interrupting a running thread. In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers. So it's not enough even for coroutines. - Yury

Hi Yury, On Tue, Apr 3, 2012 at 5:09 PM, Yury Selivanov wrote:

You shouldn't guarantee that the *whole* chain of functions/ coroutines/etc will be safe in their finally statements, you just need to protect the top coroutines in the timeouts queue.

For yield-based coroutines the common case is the following: yield lock.acquire() try: # something finally: yield lock.release() The implementation of lock.release() is something which goes to distributed locking manager, so should not be interrupted. Although I can think of some ways of fixing it using tight coupling of locks with trampoline, but having obvious way to do it without hacks is much better. (Although, I don't know how `yield from` changes working with yield-based coroutines, may be it's behavior is quite different) For greenlets situation is a bit different, as Python knows the stack there, but you still need to traverse it (or as Andrew mentioned, you can just propagate flag).

- no mechanism for interrupting a running thread.

Yes. That was intentionally to have greater chance to success. Interruption may be separate proposal.

In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers.

Which one? I know that twisted uses thread pool to handle: 1. DNS (which is just silly) 2. Few other protocols which doesn't have asynchronous libraries (which should be fixed) The whole intention of using coroutine library is to not to have thread pool. Could you describe your use case with more details?

So it's not enough even for coroutines.

Very subjective, and doesn't match my expectations. -- Paul

Hi, On Wed, Apr 4, 2012 at 4:23 AM, Yury Selivanov wrote:

On 2012-04-03, at 3:22 PM, Paul Colomiets wrote:

...

(Although, I don't know how `yield from` changes working with yield-based coroutines, may be it's behavior is quite different)

For greenlets situation is a bit different, as Python knows the stack there, but you still need to traverse it (or as Andrew mentioned, you can just propagate flag).

Why traverse?  Why propagate?  As I explained in my previous posts here, you need to protect only the top-stack coroutines in the timeouts or trampoline execution queues.  You should illustrate your logic with a more clear example - say three or four coroutines that call each other + with a glimpse of how your trampoline works. But I'm not sure that is really necessary.

Here is more detailed previous example (although, still simplified): @coroutine def add_money(user_id, money): yield redis_lock(user_id) try: yield redis_incr('user:'+user_id+':money', money) finally: yield redis_unlock(user_id) # this one is crucial to show the point of discusssion # other function are similar: @coroutine def redis_unlock(lock): yield redis_socket.wait_write() # yields back when socket is ready for writing cmd = ('DEL user:'+lock+'\n').encode('ascii') redis_socket.write(cmd) # should be loop here, actually yield redis_socket.wait_read() result = redis_socket.read(1024) # here loop too assert result == 'OK\n' The trampoline when gets coroutine from `next()` or `send()` method puts it on top of stack and doesn't dispatch original one until topmost one is exited. The point is that if timeout arrives inside a `redis_unlock` function, we must wait until finally from `add_user` is finished

...

The whole intention of using coroutine library is to not to have thread pool. Could you describe your use case with more details?

Well, our company has been using coroutines for like 2.5 years now (the framework in not yet opensourced).  And in our practice threadpool is really handy, as it allows you to:

- use non-asyncronous libraries, which you don't want to monkeypatch with greensockets (or even unable to mokeypatch)

And we rewrite them in python. It seems to be more useful.

- wrap some functions that are usually very fast, but once in a while may take some time.  And sometimes you don't want to offload them to a separate process

Ack.

- and yes, do DNS lookups if you don't have a compiled cpython extension that wraps c-ares or something alike.

Maybe let's propose asynchronous DNS library for python? We have same problem, although we do not resolve hosts at runtime (only at startup) so synchronous one is well enough for our purposes.

Please let's avoid shifting further discussion to proving or disproving the necessity of threadpools.

Agreed.

They are being actively used and there is a demand for (more or less) graceful threads interruption or abortion.

Given use cases, what stops you to make explicit interrtuption points?

Please write a PEP and we'll continue discussion from that point.  Hopefully, it will get more attention than this thread.

I don't see the point in writing a PEP until I have an idea what PEP should propose. If you have, you can do it. Again you want to implement thread interruption, and that's not my point, there is another thread for that. On Wed, Apr 4, 2012 at 3:03 AM, Greg Ewing wrote:

I don't think a frame flag on its own is quite enough. You don't just want to prevent interruptions while in a finally block, you want to defer them until the finally counter gets back to zero. Making the interrupter sleep and try again in that situation is rather ugly.

So perhaps there could also be a callback that gets invoked when the counter goes down to zero.

Do you mean put callback in a frame, which get executed at next bytecode just like signal handler, except it waits until finally clause is executed? I would work, except in may have light performance impact on each bytecode. But I'm not sure if it will be noticeable. -- Paul

Hi Yury, On Wed, Apr 4, 2012 at 7:59 PM, Yury Selivanov wrote:

...

Here is more detailed previous example (although, still simplified):

@coroutine def add_money(user_id, money):    yield redis_lock(user_id)    try:        yield redis_incr('user:'+user_id+':money', money)    finally:        yield redis_unlock(user_id)

# this one is crucial to show the point of discusssion # other function are similar: @coroutine def redis_unlock(lock):    yield redis_socket.wait_write()  # yields back when socket is ready for writing    cmd = ('DEL user:'+lock+'\n').encode('ascii')    redis_socket.write(cmd)  # should be loop here, actually    yield redis_socket.wait_read()    result = redis_socket.read(1024)  # here loop too    assert result == 'OK\n'

The trampoline when gets coroutine from `next()` or `send()` method puts it on top of stack and doesn't dispatch original one until topmost one is exited.

The point is that if timeout arrives inside a `redis_unlock` function, we must wait until finally from `add_user` is finished

How can it "arrive" inside "redis_unlock"?  Let's assume you called "add_money" as such:

yield add_money(1, 10).with_timeout(10)

Then it's the 'add_money' coroutine that should be in the tieouts queue/tree! 'add_money' specifically should be tried to be interrupted when your 10s timeout reaches.  And if 'add_money' is in its 'finally' statement - you simply postpone its interruption, meaning that 'redis_unlock' will end its execution nicely.

Again, I'm not sure how exactly you manage your timeouts.  The way I am, simplified: I have a timeouts heapq with pointers to those coroutines that were *explicitly* executed with a timeout.  So I'm protecting only the coroutines in that queue, because only them can be interrupted.  And the coroutines they call, are protected *automatically*.

If you do it differently, can you please elaborate on how your scheduler is actually designed?

I have a global timeout for processing single request. It's actually higher in a chain of generator calls. So dispatcher looks like: def dispatcher(self, method, args): with timeout(10): yield getattr(self.method)(*args) And all the local timeouts, like timeout for single request are usually applied at a socket level, where specific protocol is implemented: def redis_unlock(lock):    yield redis_socket.wait_write(2)  # wait two seconds # TimeoutError may have been raised in wait_write()    cmd = ('DEL user:'+lock+'\n').encode('ascii')    redis_socket.write(cmd)  # should be loop here, actually    yield redis_socket.wait_read(2) # another two seconds    result = redis_socket.read(1024)  # here loop too    assert result == 'OK\n' So they are not interruptions. Although, we don't use them much with coroutines, global timeout for request is usually enough. But anyway I don't see a reason to protect a single frame, because even if you have a simple mutex without coroutines you end up with: def something(): lock.acquire() try: pass finally: lock.release() And if lock's imlementation is something along the lines of: def release(self): self._native_lock.release() How would you be sure that interruption is not executed when interpreter resolved `self._native_lock.release` but not yet called it?

OK, point taken.  Please give me couple of days to at least come up with a summary document.

No hurry.

I still don't like your solution because it works directly with frames.  With an upcoming PyPy support of python 3, I don't think I want to loose the JIT support.

It's also interesting question. I don't think it's possible to interrupt JIT'ed code in arbitrary location.

Ideally, as I proposed earlier, we should introduce some sort of interruption protocol -- method 'interrupt()', with perhaps a callback.

On which object? Is it sys.interrupt()? Or is it thread.interrupt()?

...

you want to implement thread interruption, and that's not my point, there is another thread for that.

We have two requests: ability to safely interrupt python function or generator (1); ability to safely interrupt python's threads (2).  Both (1) and (2) share the same requirement of safe 'finally' statements.  To me, both features are similar enough to come up with a single solution, rather than inventing different approaches.

Again I do not propose described point (1). I propose a way to *inspect* a stack if it's safe to interrupt.

...

On Wed, Apr 4, 2012 at 3:03 AM, Greg Ewing wrote:

...

I don't think a frame flag on its own is quite enough. You don't just want to prevent interruptions while in a finally block, you want to defer them until the finally counter gets back to zero. Making the interrupter sleep and try again in that situation is rather ugly.

That's the second reason I don't like your proposal.

def foo():   try:      ..   finally:      yield unlock()   # <--- the ideal point to interrupt foo

  f = open('a', 'w')   # what if we interrupt it here?   try:      ..   finally:      f.close()

And which one fixes this problem? There is no guarantee that your timeout code haven't interrupted at " # what if we interrupt it here?". If it's a bit less likely, it's not real solution. Please, don't present it as such.

...

...

So perhaps there could also be a callback that gets invoked when the counter goes down to zero.

Do you mean put callback in a frame, which get executed at next bytecode just like signal handler, except it waits until finally clause is executed?

I would work, except in may have light performance impact on each bytecode. But I'm not sure if it will be noticeable.

That's essentially the way we currently did it.  We transform the coroutine's __code__ object to make it from:

def a():   try:      # code1   finally:      # code2

to:

def a():   __self__ = __get_current_coroutine()   try:     # code1   finally:     __self__.enter_finally()     try:       # code2     finally:       __self__.exit_finally()

'enter_finally' and 'exit_finally' maintain the internal counter of finally blocks.  If a coroutine needs to be interrupted, we check that counter.  If it is 0 - throw in a special exception.  If not - wait till it becomes 0 and throw the exception in 'exit_finally'.

The problem is in interruption of another thread. You must inspect stack only with C code holding GIL. Implementation might be more complex, but yes, it's probably can be done, without noticeable slow down. -- Paul

Hi Yury, On Wed, Apr 4, 2012 at 10:37 PM, Yury Selivanov wrote:

On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:

...

I have a global timeout for processing single request. It's actually higher in a chain of generator calls. So dispatcher looks like:

def dispatcher(self, method, args):    with timeout(10):        yield getattr(self.method)(*args)

How does it work?  To what object are you actually attaching timeout?

There is basically a "Coroutine" object. It's actually a list with paused generators, with top of them being currently running (or stopped for doing IO). It represents stack, because there is no built-in stack for generators.

And what's the advantage of having some "global" timeout instead of a timeout specifically bound to some coroutine?

We have guaranteed time of request processing. Or to be more precise guaranteed time when request stops processing so we don't have a lot of coroutines hanging forever. It allows to not to place timeouts all over the code. May be your use case is very different. E.g. this pattern doesn't work well with batch processing of big data. We process many tiny user requests per second.

Do you have that code publicly released somewhere?  I just really want to understand how exactly your architecture works to come with a better proposal (if there is one possible ;).

This framework does timeout handling in described way: https://github.com/tailhook/zorro Although, it's using greenlets. The difference is that we we don't need to keep a stack in our own scheduler when using greenlets, but everything else applies.

As an off-topic: would be interesting to have various coroutines approaches and architectures listed somewhere, to understand how python programmers actually do it.

Sure :)

...

And all the local timeouts, like timeout for single request are usually applied at a socket level, where specific protocol is implemented:

def redis_unlock(lock):    yield redis_socket.wait_write(2)  # wait two seconds   # TimeoutError may have been raised in wait_write()    cmd = ('DEL user:'+lock+'\n').encode('ascii')    redis_socket.write(cmd)  # should be loop here, actually    yield redis_socket.wait_read(2)  # another two seconds    result = redis_socket.read(1024)  # here loop too    assert result == 'OK\n'

So you have explicit timeouts in the 'redis_unlock', but you want them to be ignored if it was called from some 'finally' block?

No! I'd just omit them if I wanted. I don't want interruption of `add_money` which calls `redis_unlock` in finally to be done.

...

So they are not interruptions. Although, we don't use them much with coroutines, global timeout for request is usually enough.

Don't really follow you here.

You may think of it as socket with timeout set. socket.set_timeout(2) socket.recv(1024) It will raise TimeoutError, this should propagate as a normal exception. As opposed to being externally interrupted e.g. with SIGINT or SIGALERT.

...

But anyway I don't see a reason to protect a single frame, because even if you have a simple mutex without coroutines you end up with:

def something():  lock.acquire()  try:    pass  finally:    lock.release()

And if lock's imlementation is something along the lines of:

def release(self):    self._native_lock.release()

How would you be sure that interruption is not executed when interpreter resolved `self._native_lock.release` but not yet called it?

Is it in a context of coroutines or threads?

I don't see a difference, except the code which maintains stack. I'd say both are problem, if you neither propagate f_in_finally nor traverse a stack (although, a way of propagation may be different)

If former, the because you, perhaps, want to interrupt 'something()'?

I want to interrupt a thread. Or "Coroutine" in definition described above (having a stack of frames) or in greenlet's definition.

And it is a separate frame from the frame where 'release()' is running?

Of course (How it can be inlined? :) )

...

...

That's the second reason I don't like your proposal.

def foo():   try:      ..   finally:      yield unlock()   # <--- the ideal point to interrupt foo

  f = open('a', 'w')   # what if we interrupt it here?   try:      ..   finally:      f.close()

And which one fixes this problem? There is no guarantee that your timeout code haven't interrupted at " # what if we interrupt it here?". If it's a bit less likely, it's not real solution. Please, don't present it as such.

Sorry, I must had it explained in more details.  Right now we interrupt code only where we have a 'yield', a greenlet.switch(), or at the end of finally block, not at some arbitrary opcode.

Sure I do similar. But it doesn't work with threads, as they have no explicit yield or switch. On Wed, Apr 4, 2012 at 11:07 PM, Yury Selivanov wrote:

On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:

...

But anyway I don't see a reason to protect a single frame, because even if you have a simple mutex without coroutines you end up with:

BTW, for instance, in our framework each coroutine is a special object that wraps generator/plain function.  It controls everything that the underlying generator/function yields/returns.  But the actual execution, propagation of returned values and raised errors is the scheduler's job.  So when you are yielding a coroutine from another coroutine, frames are not even connected to each other, since the actual execution of the callee will be performed by the scheduler. It's not like a regular python call.

Same applies here. But you propagate return value/error right? So you can't say "frames are not connected". They aren't from the interpreter point of view. But they are logically connected. So for example: def a(): yield b() def b(): yield If `a().with_timeout(0.1)` is interrupted when it's waiting for value of `b()`, will `b()` continue it's execution?

For us, having 'f_in_finally' somehow propagated would be completely useless.

I hope I can convince you with this email :)

I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame.

That was original intention. But it requires stack traversing. Andrew proposed to propagate this flag, which is another point of view on the same thing (not sure which one to pick though) -- Paul

Hi Yury, On Thu, Apr 5, 2012 at 12:24 AM, Yury Selivanov wrote:

On 2012-04-04, at 4:43 PM, Paul Colomiets wrote:

...

...

How does it work?  To what object are you actually attaching timeout?

There is basically a "Coroutine" object. It's actually a list with paused generators, with top of them being currently running (or stopped for doing IO). It represents stack, because there is no built-in stack for generators.

Interesting.  I decided to go with a simple coroutine objects with a '_caller' pointer to maintain the stack virtually.

It doesn't matter. IIRC, that was to draw a tree of calls starting with roots. But that's irrelevant to the topic of discussion.

...

This framework does timeout handling in described way:

https://github.com/tailhook/zorro

Although, it's using greenlets. The difference is that we we don't need to keep a stack in our own scheduler when using greenlets, but everything else applies.

Are you using that particular framework (zorro)?  Or some modification of it that uses generators too?

Currently we experimenting with greenlets and zorro. My description of yield-based coroutines from earlier project (unfortunately non-public one).

OK, we're on the same page.  '''"frames are not connected" from the interpreter point of view''', that essentially means that 'f_in_finally' will always be a flag related only to its own frame, right?  Any 'propagation' of this flag is the responsibility of framework developers.

Yes, that's ok for me.

...

So for example:

def a():  yield b()

def b():  yield

If `a().with_timeout(0.1)` is interrupted when it's waiting for value of `b()`, will `b()` continue it's execution?

Well, in our framework, if a() is getting aborted after it's scheduled b(), but before it received the result of b(), we interrupt both of them (and those that b() might had called).

Exactly! And you don't want them to be interrupted in the case `a()` rewritten as: def a(): try: pass finally: yield b() Same with threads and greenlets.

...

...

I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame.

That was original intention. But it requires stack traversing. Andrew proposed to propagate this flag, which is another point of view on the same thing (not sure which one to pick though)

Again, if coroutines' frames aren't connected on the interpreter level (it's the responsibility of a framework), about what exact propagation are you and Andrew talking (in the sole context of the patch to cpython)?

For threads and greenlets flag can be implicitly propagated, and for yield-based coroutines f_in_finally can be made writable, so you can propagate it in you own scheduler Not sure It's good idea, just describing it. -- Paul

Paul Colomiets wrote:

Do you mean put callback in a frame, which get executed at next bytecode just like signal handler, except it waits until finally clause is executed?

It wouldn't be in each frame -- probably it would just be a global hook that gets called whenever a finally-counter anywhere gets decremented from 1 to 0. It would be passed the relevant frame so it could decide what to do from there. I don't think it would have much performance impact, since it would only get triggered by exiting a finally block. Nothing would need to happen per bytecode or anything like that. -- Greg

On 2012-04-04, at 6:45 PM, Paul Colomiets wrote:

It's similar to sys.settrace() except it only executed when finally counter decremented to 0, right? Flag `f_in_finally` is still there, right?

Yes, please keep it. With your current proposal, it's the only way to see if it is safe to interrupt coroutine right now, or we have to wait until the callback gets called.

Yury, is it ok if l'll start a PEP with this idea, and when it will have some support (or be rejected), you'll come up with thread interruption proposal?

Sure, go ahead. If I'm lucky enough to come up with a better proposal I promise to shout about it loud ;) - Yury

On Mon, Apr 2, 2012 at 3:28 PM, Yury Selivanov wrote:

On 2012-04-02, at 6:24 PM, Paul Colomiets wrote:

...

...

I still don't get how exactly do you propose to handle sudden thread interruption in your own example:

l.lock() # (!) the thread may be interrupted at this point try: ... finally: l.unlock()

You don't have a 'with' statement here.

By wrapping lock into a context manager.

How's that going to work for tons of existing code?

A context manager doesn't solve this interruption "race condition" issue anyways. If the __enter__ method is interrupted it won't have returned a context and thus __exit__ will never be called. -gps

- Yury _______________________________________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/mailman/listinfo/python-ideas