Re: [Python-ideas] Protecting finally clauses of interruptions
Hi Yury, On Tue, Apr 3, 2012 at 1:20 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
By wrapping lock into a context manager. -- Paul
Hi Yury, On Tue, Apr 3, 2012 at 1:28 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
It isn't. But it doesn't break code any more than it already is. Your proposal doesn't solve any problems with existing code too. But anyway I don't propose any new ways to interrupt code I only propose a way to inform trampoline when it's unsafe to interrupt code. -- Paul
On 2012-04-02, at 6:33 PM, Paul Colomiets wrote:
Well, if we're thinking only about interrupting coroutines (not threads), then it's going to work, yes. My initial desire to use a special exception for the purpose, was because of: - it's easier to throw exception in the thread (the C-API function already exists, and we need to think about consequences of using it) - PyPy disables the JIT when working with frames (if I recall correctly). That's why I wanted 'f_in_finally' to be an implementation detail of CPython, hidden from the user code. Perhaps PyPy could implement the handling of our special exception in a more efficient way, without the side-effect of disabling the JIT. What do you think? - Yury
Hi Yury, On Tue, Apr 3, 2012 at 2:04 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Yes the threading stuff is more complex. For the main thread there are few possible implementations, e.g. using signals. If thread signals were ever implemented in python they can be used too. The real problem is inspecting a stack from another thread. But my solution by itself gives a pretty big field of experimentation. Like you can wrap every blocking call, and check the stack on EINTR, and either send an exception or wait a bit, like with coroutines (and you can emit EINTR with pthread_kill, and implement waiting either using another thread or using sys.settrace(), as I don't think performance really matter here)
It's nice for python to have finally protection built-in, but I don't see how it can be implemented in a generic way. E.g. I usually don't want to break finally unless it executes too long, or if I hit Ctrl+C multiple times. And that is too subjective, to be proposed as common behavior. I also doubt how it can work with stack of yield-based coroutines, as it knows nothing about this kind of stack.
Yes, I think PyPy can make some exception. Or maybe until PyPy implement support of 3.3, some library may end up with nice high level API, which both python implementation can include. -- Paul
On 2012-04-02, at 7:36 PM, Paul Colomiets wrote:
It's nice for python to have finally protection built-in, but I don't see how it can be implemented in a generic way.
How about adding some sort of 'interruption protocol'? Say Threads, generators, and Greenlets will have a special method called '_interrupt'. While it will be implemented differently for each of them, it will work on the same principle: - check if the underlying code block is in one of its 'finally' statements - if it is: set a special flag to abort when the frame's counter of 'finally' blocks reaches 0 to raise the ExecutionInterrupt exception - if it is not: raise the ExecutionInterrupt exception right away - Yury
Hi Yury, On Tue, Apr 3, 2012 at 3:02 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
On the first look, it's nice proposal. On the second one we have the following problems: 1. For yield-based coroutines you must inspect stack anyway, since interpreter doesn't have a stack, you build it yourself (although, I don't know how `yield from` changes that) 2. For greenlet based coroutines it is unclear what the stack is. For example: def f1(): try: pass finally: g1.switch() def f2(): sleep(1.0) g1 = greenlet(f1) g2 = greenlet(f2) g1.switch() Is it safe to interrupt g2 while it's in `sleep`? (If you wonder how I fix this problem with f_in_finally stack, it's easy. I usually switch to a coroutine from trampoline, so this is a boundary of the stack which should be checked for f_in_finally). 3. For threads it was discussed several times and rejected. This proposal may make thread interruptions slightly safer, but I'm not sure it's enough to convince people. Also at the first implementation we may oversight some places where it's unsafe to break. Like for some objects __init__/__exit__ is safe pair of functions not __enter__/__exit__. So we might need `with` expression to be uninterrtuptable, for the code like: with open('something') as f: ... It may break if interrupted inside `open()`. (Although, if __enter__ is protected you can fix the problem with a simple wrapper). So I still propose add a frame flag, which doesn't break anything, and gives us experiment with interruptions without putting some experimental code into the core. -- Paul
On 2012-04-03, at 3:16 AM, Paul Colomiets wrote:
Wait. So you're tracing the whole coroutine execution stack to check if the current coroutine was called in a finally block of some other coroutine? For handling timeouts I don't think that is necessary (maybe there are other use cases?) In the example below you actually have to interrupt g2: def g1(): try: ... finally: g2().with_timeout(0.1) def g2(): sleep(2) You shouldn't guarantee that the *whole* chain of functions/ coroutines/etc will be safe in their finally statements, you just need to protect the top coroutines in the timeouts queue. Hence, in the above example, if you run g1() with a timeout, the trampoline should ensure that it won't interrupt it while it is in its finally block. But it can interrupt g2() in any context at any point of its execution. And if g2() gets interrupted, g1()'s finally statement will be broken, yes. But that's the responsibility of the developer to ensure that the code in 'finally' handles exceptions within it correctly. That's just my approach to handle timeouts, I'm not advocating it to be the very right one. Are there any other use-cases when you have to inspect the execution stack? Because if there is no, 'interrupt()' method is sufficient and implementable, as both generators and greenlets are well aware about the code frames they holding.
That's why I'm advocating for a PEP. Thread interruption isn't a safe feature in the .NET CLR either. You may break things with it there too. And it doesn't protect the chain of functions calling each other from their 'finally' statements, it just protects the top frame. The 'abort' and 'interrupt' methods aren't advertised to be used in .NET, use them at your own risk. So I don't think that we can, or should ensure 100% safety when interrupting a thread. And that's why I think it is worth to propose a mechanism that will work for many concurrency primitives.
There are cons and pros in your solution. Pros ---- - can be used right away in coroutine libraries. - somewhat simple and small CPython patch. Cons ---- - you have to work with frames almost throughout the execution of the program. In PyPy you simply will have the JIT disabled. And I'm not sure how frame access works in Jython and IronPython from the performance point of view. - no mechanism for interrupting a running thread. In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers. So it's not enough even for coroutines. - Yury
Hi Yury, On Tue, Apr 3, 2012 at 5:09 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
For yield-based coroutines the common case is the following: yield lock.acquire() try: # something finally: yield lock.release() The implementation of lock.release() is something which goes to distributed locking manager, so should not be interrupted. Although I can think of some ways of fixing it using tight coupling of locks with trampoline, but having obvious way to do it without hacks is much better. (Although, I don't know how `yield from` changes working with yield-based coroutines, may be it's behavior is quite different) For greenlets situation is a bit different, as Python knows the stack there, but you still need to traverse it (or as Andrew mentioned, you can just propagate flag).
- no mechanism for interrupting a running thread.
Yes. That was intentionally to have greater chance to success. Interruption may be separate proposal.
In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers.
Which one? I know that twisted uses thread pool to handle: 1. DNS (which is just silly) 2. Few other protocols which doesn't have asynchronous libraries (which should be fixed) The whole intention of using coroutine library is to not to have thread pool. Could you describe your use case with more details?
So it's not enough even for coroutines.
Very subjective, and doesn't match my expectations. -- Paul
On 2012-04-03, at 3:22 PM, Paul Colomiets wrote:
Why traverse? Why propagate? As I explained in my previous posts here, you need to protect only the top-stack coroutines in the timeouts or trampoline execution queues. You should illustrate your logic with a more clear example - say three or four coroutines that call each other + with a glimpse of how your trampoline works. But I'm not sure that is really necessary.
Besides Twisted? eventlet; gevent will have them in 1.0, etc.
Well, our company has been using coroutines for like 2.5 years now (the framework in not yet opensourced). And in our practice threadpool is really handy, as it allows you to: - use non-asyncronous libraries, which you don't want to monkeypatch with greensockets (or even unable to mokeypatch) - wrap some functions that are usually very fast, but once in a while may take some time. And sometimes you don't want to offload them to a separate process - and yes, do DNS lookups if you don't have a compiled cpython extension that wraps c-ares or something alike. Please let's avoid shifting further discussion to proving or disproving the necessity of threadpools. They are being actively used and there is a demand for (more or less) graceful threads interruption or abortion.
So it's not enough even for coroutines.
Very subjective, and doesn't match my expectations.
As I said -- we've been working with coroutines (combined generators + greenlets) for a few years, and apparently have different experience, opinions and expectations from what you have. And I suppose developers and users of eventlet, gevent, twisted (@inlineCallbacks) and other libraries have their own opinions and ideas too. Not to mention, it would be interesting to hear from PyPy, Juthon and IronPython teams. It also seems that neither of us have enough experience working with 'yield from' style coroutines. Please write a PEP and we'll continue discussion from that point. Hopefully, it will get more attention than this thread. - Yury
Hi, On Wed, Apr 4, 2012 at 4:23 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Here is more detailed previous example (although, still simplified): @coroutine def add_money(user_id, money): yield redis_lock(user_id) try: yield redis_incr('user:'+user_id+':money', money) finally: yield redis_unlock(user_id) # this one is crucial to show the point of discusssion # other function are similar: @coroutine def redis_unlock(lock): yield redis_socket.wait_write() # yields back when socket is ready for writing cmd = ('DEL user:'+lock+'\n').encode('ascii') redis_socket.write(cmd) # should be loop here, actually yield redis_socket.wait_read() result = redis_socket.read(1024) # here loop too assert result == 'OK\n' The trampoline when gets coroutine from `next()` or `send()` method puts it on top of stack and doesn't dispatch original one until topmost one is exited. The point is that if timeout arrives inside a `redis_unlock` function, we must wait until finally from `add_user` is finished
And we rewrite them in python. It seems to be more useful.
Ack.
- and yes, do DNS lookups if you don't have a compiled cpython extension that wraps c-ares or something alike.
Maybe let's propose asynchronous DNS library for python? We have same problem, although we do not resolve hosts at runtime (only at startup) so synchronous one is well enough for our purposes.
Please let's avoid shifting further discussion to proving or disproving the necessity of threadpools.
Agreed.
They are being actively used and there is a demand for (more or less) graceful threads interruption or abortion.
Given use cases, what stops you to make explicit interrtuption points?
Please write a PEP and we'll continue discussion from that point. Hopefully, it will get more attention than this thread.
I don't see the point in writing a PEP until I have an idea what PEP should propose. If you have, you can do it. Again you want to implement thread interruption, and that's not my point, there is another thread for that. On Wed, Apr 4, 2012 at 3:03 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Do you mean put callback in a frame, which get executed at next bytecode just like signal handler, except it waits until finally clause is executed? I would work, except in may have light performance impact on each bytecode. But I'm not sure if it will be noticeable. -- Paul
On 2012-04-04, at 4:04 AM, Paul Colomiets wrote:
How can it "arrive" inside "redis_unlock"? Let's assume you called "add_money" as such: yield add_money(1, 10).with_timeout(10) Then it's the 'add_money' coroutine that should be in the tieouts queue/tree! 'add_money' specifically should be tried to be interrupted when your 10s timeout reaches. And if 'add_money' is in its 'finally' statement - you simply postpone its interruption, meaning that 'redis_unlock' will end its execution nicely. Again, I'm not sure how exactly you manage your timeouts. The way I am, simplified: I have a timeouts heapq with pointers to those coroutines that were *explicitly* executed with a timeout. So I'm protecting only the coroutines in that queue, because only them can be interrupted. And the coroutines they call, are protected *automatically*. If you do it differently, can you please elaborate on how your scheduler is actually designed?
Sometimes you can't afford the luxury ;)
OK, point taken. Please give me couple of days to at least come up with a summary document. I still don't like your solution because it works directly with frames. With an upcoming PyPy support of python 3, I don't think I want to loose the JIT support. I also want to take a look at the new PyPy continuations. Ideally, as I proposed earlier, we should introduce some sort of interruption protocol -- method 'interrupt()', with perhaps a callback.
you want to implement thread interruption, and that's not my point, there is another thread for that.
We have two requests: ability to safely interrupt python function or generator (1); ability to safely interrupt python's threads (2). Both (1) and (2) share the same requirement of safe 'finally' statements. To me, both features are similar enough to come up with a single solution, rather than inventing different approaches.
That's the second reason I don't like your proposal. def foo(): try: .. finally: yield unlock() # <--- the ideal point to interrupt foo f = open('a', 'w') # what if we interrupt it here? try: .. finally: f.close()
That's essentially the way we currently did it. We transform the coroutine's __code__ object to make it from: def a(): try: # code1 finally: # code2 to: def a(): __self__ = __get_current_coroutine() try: # code1 finally: __self__.enter_finally() try: # code2 finally: __self__.exit_finally() 'enter_finally' and 'exit_finally' maintain the internal counter of finally blocks. If a coroutine needs to be interrupted, we check that counter. If it is 0 - throw in a special exception. If not - wait till it becomes 0 and throw the exception in 'exit_finally'. Works flawlessly, but with the high cost of having to patch __code__ objects. - Yury
Hi Yury, On Wed, Apr 4, 2012 at 7:59 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
I have a global timeout for processing single request. It's actually higher in a chain of generator calls. So dispatcher looks like: def dispatcher(self, method, args): with timeout(10): yield getattr(self.method)(*args) And all the local timeouts, like timeout for single request are usually applied at a socket level, where specific protocol is implemented: def redis_unlock(lock): yield redis_socket.wait_write(2) # wait two seconds # TimeoutError may have been raised in wait_write() cmd = ('DEL user:'+lock+'\n').encode('ascii') redis_socket.write(cmd) # should be loop here, actually yield redis_socket.wait_read(2) # another two seconds result = redis_socket.read(1024) # here loop too assert result == 'OK\n' So they are not interruptions. Although, we don't use them much with coroutines, global timeout for request is usually enough. But anyway I don't see a reason to protect a single frame, because even if you have a simple mutex without coroutines you end up with: def something(): lock.acquire() try: pass finally: lock.release() And if lock's imlementation is something along the lines of: def release(self): self._native_lock.release() How would you be sure that interruption is not executed when interpreter resolved `self._native_lock.release` but not yet called it?
OK, point taken. Please give me couple of days to at least come up with a summary document.
No hurry.
It's also interesting question. I don't think it's possible to interrupt JIT'ed code in arbitrary location.
On which object? Is it sys.interrupt()? Or is it thread.interrupt()?
Again I do not propose described point (1). I propose a way to *inspect* a stack if it's safe to interrupt.
And which one fixes this problem? There is no guarantee that your timeout code haven't interrupted at " # what if we interrupt it here?". If it's a bit less likely, it's not real solution. Please, don't present it as such.
The problem is in interruption of another thread. You must inspect stack only with C code holding GIL. Implementation might be more complex, but yes, it's probably can be done, without noticeable slow down. -- Paul
On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:
How does it work? To what object are you actually attaching timeout? I'm just curious now how your 'timeout' context manager works. And what's the advantage of having some "global" timeout instead of a timeout specifically bound to some coroutine? Do you have that code publicly released somewhere? I just really want to understand how exactly your architecture works to come with a better proposal (if there is one possible ;). As an off-topic: would be interesting to have various coroutines approaches and architectures listed somewhere, to understand how python programmers actually do it.
So you have explicit timeouts in the 'redis_unlock', but you want them to be ignored if it was called from some 'finally' block?
Don't really follow you here.
Is it in a context of coroutines or threads? If former, then because you, perhaps, want to interrupt 'something()'? And it is a separate frame from the frame where 'release()' is running?
It's also interesting question. I don't think it's possible to interrupt JIT'ed code in arbitrary location.
I guess that should really be asked on the pypy-dev mail-list, once we have a proposal.
Sorry, I must had it explained in more details. Right now we interrupt code only where we have a 'yield', a greenlet.switch(), or at the end of finally block, not at some arbitrary opcode. - Yury
Hi Yury, On Wed, Apr 4, 2012 at 10:37 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
There is basically a "Coroutine" object. It's actually a list with paused generators, with top of them being currently running (or stopped for doing IO). It represents stack, because there is no built-in stack for generators.
And what's the advantage of having some "global" timeout instead of a timeout specifically bound to some coroutine?
We have guaranteed time of request processing. Or to be more precise guaranteed time when request stops processing so we don't have a lot of coroutines hanging forever. It allows to not to place timeouts all over the code. May be your use case is very different. E.g. this pattern doesn't work well with batch processing of big data. We process many tiny user requests per second.
This framework does timeout handling in described way: https://github.com/tailhook/zorro Although, it's using greenlets. The difference is that we we don't need to keep a stack in our own scheduler when using greenlets, but everything else applies.
Sure :)
No! I'd just omit them if I wanted. I don't want interruption of `add_money` which calls `redis_unlock` in finally to be done.
You may think of it as socket with timeout set. socket.set_timeout(2) socket.recv(1024) It will raise TimeoutError, this should propagate as a normal exception. As opposed to being externally interrupted e.g. with SIGINT or SIGALERT.
I don't see a difference, except the code which maintains stack. I'd say both are problem, if you neither propagate f_in_finally nor traverse a stack (although, a way of propagation may be different)
If former, the because you, perhaps, want to interrupt 'something()'?
I want to interrupt a thread. Or "Coroutine" in definition described above (having a stack of frames) or in greenlet's definition.
And it is a separate frame from the frame where 'release()' is running?
Of course (How it can be inlined? :) )
Sure I do similar. But it doesn't work with threads, as they have no explicit yield or switch. On Wed, Apr 4, 2012 at 11:07 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Same applies here. But you propagate return value/error right? So you can't say "frames are not connected". They aren't from the interpreter point of view. But they are logically connected. So for example: def a(): yield b() def b(): yield If `a().with_timeout(0.1)` is interrupted when it's waiting for value of `b()`, will `b()` continue it's execution?
For us, having 'f_in_finally' somehow propagated would be completely useless.
I hope I can convince you with this email :)
I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame.
That was original intention. But it requires stack traversing. Andrew proposed to propagate this flag, which is another point of view on the same thing (not sure which one to pick though) -- Paul
On 2012-04-04, at 4:43 PM, Paul Colomiets wrote:
Interesting. I decided to go with a simple coroutine objects with a '_caller' pointer to maintain the stack virtually.
Are you using that particular framework (zorro)? Or some modification of it that uses generators too?
OK.
OK, we're on the same page. '''"frames are not connected" from the interpreter point of view''', that essentially means that 'f_in_finally' will always be a flag related only to its own frame, right? Any 'propagation' of this flag is the responsibility of framework developers.
Well, in our framework, if a() is getting aborted after it's scheduled b(), but before it received the result of b(), we interrupt both of them (and those that b() might had called).
Again, if coroutines' frames aren't connected on the interpreter level (it's the responsibility of a framework), about what exact propagation are you and Andrew talking (in the sole context of the patch to cpython)? - Yury
Hi Yury, On Thu, Apr 5, 2012 at 12:24 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
It doesn't matter. IIRC, that was to draw a tree of calls starting with roots. But that's irrelevant to the topic of discussion.
Currently we experimenting with greenlets and zorro. My description of yield-based coroutines from earlier project (unfortunately non-public one).
Yes, that's ok for me.
Exactly! And you don't want them to be interrupted in the case `a()` rewritten as: def a(): try: pass finally: yield b() Same with threads and greenlets.
For threads and greenlets flag can be implicitly propagated, and for yield-based coroutines f_in_finally can be made writable, so you can propagate it in you own scheduler Not sure It's good idea, just describing it. -- Paul
On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:
BTW, for instance, in our framework each coroutine is a special object that wraps generator/plain function. It controls everything that the underlying generator/function yields/returns. But the actual execution, propagation of returned values and raised errors is the scheduler's job. So when you are yielding a coroutine from another coroutine, frames are not even connected to each other, since the actual execution of the callee will be performed by the scheduler. It's not like a regular python call. For us, having 'f_in_finally' somehow propagated would be completely useless. I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame. - Yury
Paul Colomiets wrote:
It wouldn't be in each frame -- probably it would just be a global hook that gets called whenever a finally-counter anywhere gets decremented from 1 to 0. It would be passed the relevant frame so it could decide what to do from there. I don't think it would have much performance impact, since it would only get triggered by exiting a finally block. Nothing would need to happen per bytecode or anything like that. -- Greg
Hi Greg, On Thu, Apr 5, 2012 at 1:18 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
It's similar to sys.settrace() except it only executed when finally counter decremented to 0, right? Flag `f_in_finally` is still there, right? It solves my use case well. Yury, is it ok if l'll start a PEP with this idea, and when it will have some support (or be rejected), you'll come up with thread interruption proposal? -- Paul
On 2012-04-04, at 6:45 PM, Paul Colomiets wrote:
Yes, please keep it. With your current proposal, it's the only way to see if it is safe to interrupt coroutine right now, or we have to wait until the callback gets called.
Sure, go ahead. If I'm lucky enough to come up with a better proposal I promise to shout about it loud ;) - Yury
Paul Colomiets wrote:
I don't think a frame flag on its own is quite enough. You don't just want to prevent interruptions while in a finally block, you want to defer them until the finally counter gets back to zero. Making the interrupter sleep and try again in that situation is rather ugly. So perhaps there could also be a callback that gets invoked when the counter goes down to zero. -- Greg
On Mon, Apr 2, 2012 at 3:28 PM, Yury Selivanov <yselivanov.ml@gmail.com>wrote:
A context manager doesn't solve this interruption "race condition" issue anyways. If the __enter__ method is interrupted it won't have returned a context and thus __exit__ will never be called. -gps
On 2012-04-02, at 7:10 PM, Gregory P. Smith wrote:
If the __enter__ method is interrupted it won't have returned a context and thus __exit__ will never be called.
To address that Paul proposed to make __enter__ non-interruptable as well. - Yury
Hi Yury, On Tue, Apr 3, 2012 at 1:28 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
It isn't. But it doesn't break code any more than it already is. Your proposal doesn't solve any problems with existing code too. But anyway I don't propose any new ways to interrupt code I only propose a way to inform trampoline when it's unsafe to interrupt code. -- Paul
On 2012-04-02, at 6:33 PM, Paul Colomiets wrote:
Well, if we're thinking only about interrupting coroutines (not threads), then it's going to work, yes. My initial desire to use a special exception for the purpose, was because of: - it's easier to throw exception in the thread (the C-API function already exists, and we need to think about consequences of using it) - PyPy disables the JIT when working with frames (if I recall correctly). That's why I wanted 'f_in_finally' to be an implementation detail of CPython, hidden from the user code. Perhaps PyPy could implement the handling of our special exception in a more efficient way, without the side-effect of disabling the JIT. What do you think? - Yury
Hi Yury, On Tue, Apr 3, 2012 at 2:04 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Yes the threading stuff is more complex. For the main thread there are few possible implementations, e.g. using signals. If thread signals were ever implemented in python they can be used too. The real problem is inspecting a stack from another thread. But my solution by itself gives a pretty big field of experimentation. Like you can wrap every blocking call, and check the stack on EINTR, and either send an exception or wait a bit, like with coroutines (and you can emit EINTR with pthread_kill, and implement waiting either using another thread or using sys.settrace(), as I don't think performance really matter here)
It's nice for python to have finally protection built-in, but I don't see how it can be implemented in a generic way. E.g. I usually don't want to break finally unless it executes too long, or if I hit Ctrl+C multiple times. And that is too subjective, to be proposed as common behavior. I also doubt how it can work with stack of yield-based coroutines, as it knows nothing about this kind of stack.
Yes, I think PyPy can make some exception. Or maybe until PyPy implement support of 3.3, some library may end up with nice high level API, which both python implementation can include. -- Paul
On 2012-04-02, at 7:36 PM, Paul Colomiets wrote:
It's nice for python to have finally protection built-in, but I don't see how it can be implemented in a generic way.
How about adding some sort of 'interruption protocol'? Say Threads, generators, and Greenlets will have a special method called '_interrupt'. While it will be implemented differently for each of them, it will work on the same principle: - check if the underlying code block is in one of its 'finally' statements - if it is: set a special flag to abort when the frame's counter of 'finally' blocks reaches 0 to raise the ExecutionInterrupt exception - if it is not: raise the ExecutionInterrupt exception right away - Yury
Hi Yury, On Tue, Apr 3, 2012 at 3:02 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
On the first look, it's nice proposal. On the second one we have the following problems: 1. For yield-based coroutines you must inspect stack anyway, since interpreter doesn't have a stack, you build it yourself (although, I don't know how `yield from` changes that) 2. For greenlet based coroutines it is unclear what the stack is. For example: def f1(): try: pass finally: g1.switch() def f2(): sleep(1.0) g1 = greenlet(f1) g2 = greenlet(f2) g1.switch() Is it safe to interrupt g2 while it's in `sleep`? (If you wonder how I fix this problem with f_in_finally stack, it's easy. I usually switch to a coroutine from trampoline, so this is a boundary of the stack which should be checked for f_in_finally). 3. For threads it was discussed several times and rejected. This proposal may make thread interruptions slightly safer, but I'm not sure it's enough to convince people. Also at the first implementation we may oversight some places where it's unsafe to break. Like for some objects __init__/__exit__ is safe pair of functions not __enter__/__exit__. So we might need `with` expression to be uninterrtuptable, for the code like: with open('something') as f: ... It may break if interrupted inside `open()`. (Although, if __enter__ is protected you can fix the problem with a simple wrapper). So I still propose add a frame flag, which doesn't break anything, and gives us experiment with interruptions without putting some experimental code into the core. -- Paul
On 2012-04-03, at 3:16 AM, Paul Colomiets wrote:
Wait. So you're tracing the whole coroutine execution stack to check if the current coroutine was called in a finally block of some other coroutine? For handling timeouts I don't think that is necessary (maybe there are other use cases?) In the example below you actually have to interrupt g2: def g1(): try: ... finally: g2().with_timeout(0.1) def g2(): sleep(2) You shouldn't guarantee that the *whole* chain of functions/ coroutines/etc will be safe in their finally statements, you just need to protect the top coroutines in the timeouts queue. Hence, in the above example, if you run g1() with a timeout, the trampoline should ensure that it won't interrupt it while it is in its finally block. But it can interrupt g2() in any context at any point of its execution. And if g2() gets interrupted, g1()'s finally statement will be broken, yes. But that's the responsibility of the developer to ensure that the code in 'finally' handles exceptions within it correctly. That's just my approach to handle timeouts, I'm not advocating it to be the very right one. Are there any other use-cases when you have to inspect the execution stack? Because if there is no, 'interrupt()' method is sufficient and implementable, as both generators and greenlets are well aware about the code frames they holding.
That's why I'm advocating for a PEP. Thread interruption isn't a safe feature in the .NET CLR either. You may break things with it there too. And it doesn't protect the chain of functions calling each other from their 'finally' statements, it just protects the top frame. The 'abort' and 'interrupt' methods aren't advertised to be used in .NET, use them at your own risk. So I don't think that we can, or should ensure 100% safety when interrupting a thread. And that's why I think it is worth to propose a mechanism that will work for many concurrency primitives.
There are cons and pros in your solution. Pros ---- - can be used right away in coroutine libraries. - somewhat simple and small CPython patch. Cons ---- - you have to work with frames almost throughout the execution of the program. In PyPy you simply will have the JIT disabled. And I'm not sure how frame access works in Jython and IronPython from the performance point of view. - no mechanism for interrupting a running thread. In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers. So it's not enough even for coroutines. - Yury
Hi Yury, On Tue, Apr 3, 2012 at 5:09 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
For yield-based coroutines the common case is the following: yield lock.acquire() try: # something finally: yield lock.release() The implementation of lock.release() is something which goes to distributed locking manager, so should not be interrupted. Although I can think of some ways of fixing it using tight coupling of locks with trampoline, but having obvious way to do it without hacks is much better. (Although, I don't know how `yield from` changes working with yield-based coroutines, may be it's behavior is quite different) For greenlets situation is a bit different, as Python knows the stack there, but you still need to traverse it (or as Andrew mentioned, you can just propagate flag).
- no mechanism for interrupting a running thread.
Yes. That was intentionally to have greater chance to success. Interruption may be separate proposal.
In almost any coroutine library you will have a thread pool, and sometimes you need a way to interrupt workers.
Which one? I know that twisted uses thread pool to handle: 1. DNS (which is just silly) 2. Few other protocols which doesn't have asynchronous libraries (which should be fixed) The whole intention of using coroutine library is to not to have thread pool. Could you describe your use case with more details?
So it's not enough even for coroutines.
Very subjective, and doesn't match my expectations. -- Paul
On 2012-04-03, at 3:22 PM, Paul Colomiets wrote:
Why traverse? Why propagate? As I explained in my previous posts here, you need to protect only the top-stack coroutines in the timeouts or trampoline execution queues. You should illustrate your logic with a more clear example - say three or four coroutines that call each other + with a glimpse of how your trampoline works. But I'm not sure that is really necessary.
Besides Twisted? eventlet; gevent will have them in 1.0, etc.
Well, our company has been using coroutines for like 2.5 years now (the framework in not yet opensourced). And in our practice threadpool is really handy, as it allows you to: - use non-asyncronous libraries, which you don't want to monkeypatch with greensockets (or even unable to mokeypatch) - wrap some functions that are usually very fast, but once in a while may take some time. And sometimes you don't want to offload them to a separate process - and yes, do DNS lookups if you don't have a compiled cpython extension that wraps c-ares or something alike. Please let's avoid shifting further discussion to proving or disproving the necessity of threadpools. They are being actively used and there is a demand for (more or less) graceful threads interruption or abortion.
So it's not enough even for coroutines.
Very subjective, and doesn't match my expectations.
As I said -- we've been working with coroutines (combined generators + greenlets) for a few years, and apparently have different experience, opinions and expectations from what you have. And I suppose developers and users of eventlet, gevent, twisted (@inlineCallbacks) and other libraries have their own opinions and ideas too. Not to mention, it would be interesting to hear from PyPy, Juthon and IronPython teams. It also seems that neither of us have enough experience working with 'yield from' style coroutines. Please write a PEP and we'll continue discussion from that point. Hopefully, it will get more attention than this thread. - Yury
Hi, On Wed, Apr 4, 2012 at 4:23 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Here is more detailed previous example (although, still simplified): @coroutine def add_money(user_id, money): yield redis_lock(user_id) try: yield redis_incr('user:'+user_id+':money', money) finally: yield redis_unlock(user_id) # this one is crucial to show the point of discusssion # other function are similar: @coroutine def redis_unlock(lock): yield redis_socket.wait_write() # yields back when socket is ready for writing cmd = ('DEL user:'+lock+'\n').encode('ascii') redis_socket.write(cmd) # should be loop here, actually yield redis_socket.wait_read() result = redis_socket.read(1024) # here loop too assert result == 'OK\n' The trampoline when gets coroutine from `next()` or `send()` method puts it on top of stack and doesn't dispatch original one until topmost one is exited. The point is that if timeout arrives inside a `redis_unlock` function, we must wait until finally from `add_user` is finished
And we rewrite them in python. It seems to be more useful.
Ack.
- and yes, do DNS lookups if you don't have a compiled cpython extension that wraps c-ares or something alike.
Maybe let's propose asynchronous DNS library for python? We have same problem, although we do not resolve hosts at runtime (only at startup) so synchronous one is well enough for our purposes.
Please let's avoid shifting further discussion to proving or disproving the necessity of threadpools.
Agreed.
They are being actively used and there is a demand for (more or less) graceful threads interruption or abortion.
Given use cases, what stops you to make explicit interrtuption points?
Please write a PEP and we'll continue discussion from that point. Hopefully, it will get more attention than this thread.
I don't see the point in writing a PEP until I have an idea what PEP should propose. If you have, you can do it. Again you want to implement thread interruption, and that's not my point, there is another thread for that. On Wed, Apr 4, 2012 at 3:03 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
Do you mean put callback in a frame, which get executed at next bytecode just like signal handler, except it waits until finally clause is executed? I would work, except in may have light performance impact on each bytecode. But I'm not sure if it will be noticeable. -- Paul
On 2012-04-04, at 4:04 AM, Paul Colomiets wrote:
How can it "arrive" inside "redis_unlock"? Let's assume you called "add_money" as such: yield add_money(1, 10).with_timeout(10) Then it's the 'add_money' coroutine that should be in the tieouts queue/tree! 'add_money' specifically should be tried to be interrupted when your 10s timeout reaches. And if 'add_money' is in its 'finally' statement - you simply postpone its interruption, meaning that 'redis_unlock' will end its execution nicely. Again, I'm not sure how exactly you manage your timeouts. The way I am, simplified: I have a timeouts heapq with pointers to those coroutines that were *explicitly* executed with a timeout. So I'm protecting only the coroutines in that queue, because only them can be interrupted. And the coroutines they call, are protected *automatically*. If you do it differently, can you please elaborate on how your scheduler is actually designed?
Sometimes you can't afford the luxury ;)
OK, point taken. Please give me couple of days to at least come up with a summary document. I still don't like your solution because it works directly with frames. With an upcoming PyPy support of python 3, I don't think I want to loose the JIT support. I also want to take a look at the new PyPy continuations. Ideally, as I proposed earlier, we should introduce some sort of interruption protocol -- method 'interrupt()', with perhaps a callback.
you want to implement thread interruption, and that's not my point, there is another thread for that.
We have two requests: ability to safely interrupt python function or generator (1); ability to safely interrupt python's threads (2). Both (1) and (2) share the same requirement of safe 'finally' statements. To me, both features are similar enough to come up with a single solution, rather than inventing different approaches.
That's the second reason I don't like your proposal. def foo(): try: .. finally: yield unlock() # <--- the ideal point to interrupt foo f = open('a', 'w') # what if we interrupt it here? try: .. finally: f.close()
That's essentially the way we currently did it. We transform the coroutine's __code__ object to make it from: def a(): try: # code1 finally: # code2 to: def a(): __self__ = __get_current_coroutine() try: # code1 finally: __self__.enter_finally() try: # code2 finally: __self__.exit_finally() 'enter_finally' and 'exit_finally' maintain the internal counter of finally blocks. If a coroutine needs to be interrupted, we check that counter. If it is 0 - throw in a special exception. If not - wait till it becomes 0 and throw the exception in 'exit_finally'. Works flawlessly, but with the high cost of having to patch __code__ objects. - Yury
Hi Yury, On Wed, Apr 4, 2012 at 7:59 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
I have a global timeout for processing single request. It's actually higher in a chain of generator calls. So dispatcher looks like: def dispatcher(self, method, args): with timeout(10): yield getattr(self.method)(*args) And all the local timeouts, like timeout for single request are usually applied at a socket level, where specific protocol is implemented: def redis_unlock(lock): yield redis_socket.wait_write(2) # wait two seconds # TimeoutError may have been raised in wait_write() cmd = ('DEL user:'+lock+'\n').encode('ascii') redis_socket.write(cmd) # should be loop here, actually yield redis_socket.wait_read(2) # another two seconds result = redis_socket.read(1024) # here loop too assert result == 'OK\n' So they are not interruptions. Although, we don't use them much with coroutines, global timeout for request is usually enough. But anyway I don't see a reason to protect a single frame, because even if you have a simple mutex without coroutines you end up with: def something(): lock.acquire() try: pass finally: lock.release() And if lock's imlementation is something along the lines of: def release(self): self._native_lock.release() How would you be sure that interruption is not executed when interpreter resolved `self._native_lock.release` but not yet called it?
OK, point taken. Please give me couple of days to at least come up with a summary document.
No hurry.
It's also interesting question. I don't think it's possible to interrupt JIT'ed code in arbitrary location.
On which object? Is it sys.interrupt()? Or is it thread.interrupt()?
Again I do not propose described point (1). I propose a way to *inspect* a stack if it's safe to interrupt.
And which one fixes this problem? There is no guarantee that your timeout code haven't interrupted at " # what if we interrupt it here?". If it's a bit less likely, it's not real solution. Please, don't present it as such.
The problem is in interruption of another thread. You must inspect stack only with C code holding GIL. Implementation might be more complex, but yes, it's probably can be done, without noticeable slow down. -- Paul
On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:
How does it work? To what object are you actually attaching timeout? I'm just curious now how your 'timeout' context manager works. And what's the advantage of having some "global" timeout instead of a timeout specifically bound to some coroutine? Do you have that code publicly released somewhere? I just really want to understand how exactly your architecture works to come with a better proposal (if there is one possible ;). As an off-topic: would be interesting to have various coroutines approaches and architectures listed somewhere, to understand how python programmers actually do it.
So you have explicit timeouts in the 'redis_unlock', but you want them to be ignored if it was called from some 'finally' block?
Don't really follow you here.
Is it in a context of coroutines or threads? If former, then because you, perhaps, want to interrupt 'something()'? And it is a separate frame from the frame where 'release()' is running?
It's also interesting question. I don't think it's possible to interrupt JIT'ed code in arbitrary location.
I guess that should really be asked on the pypy-dev mail-list, once we have a proposal.
Sorry, I must had it explained in more details. Right now we interrupt code only where we have a 'yield', a greenlet.switch(), or at the end of finally block, not at some arbitrary opcode. - Yury
Hi Yury, On Wed, Apr 4, 2012 at 10:37 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
There is basically a "Coroutine" object. It's actually a list with paused generators, with top of them being currently running (or stopped for doing IO). It represents stack, because there is no built-in stack for generators.
And what's the advantage of having some "global" timeout instead of a timeout specifically bound to some coroutine?
We have guaranteed time of request processing. Or to be more precise guaranteed time when request stops processing so we don't have a lot of coroutines hanging forever. It allows to not to place timeouts all over the code. May be your use case is very different. E.g. this pattern doesn't work well with batch processing of big data. We process many tiny user requests per second.
This framework does timeout handling in described way: https://github.com/tailhook/zorro Although, it's using greenlets. The difference is that we we don't need to keep a stack in our own scheduler when using greenlets, but everything else applies.
Sure :)
No! I'd just omit them if I wanted. I don't want interruption of `add_money` which calls `redis_unlock` in finally to be done.
You may think of it as socket with timeout set. socket.set_timeout(2) socket.recv(1024) It will raise TimeoutError, this should propagate as a normal exception. As opposed to being externally interrupted e.g. with SIGINT or SIGALERT.
I don't see a difference, except the code which maintains stack. I'd say both are problem, if you neither propagate f_in_finally nor traverse a stack (although, a way of propagation may be different)
If former, the because you, perhaps, want to interrupt 'something()'?
I want to interrupt a thread. Or "Coroutine" in definition described above (having a stack of frames) or in greenlet's definition.
And it is a separate frame from the frame where 'release()' is running?
Of course (How it can be inlined? :) )
Sure I do similar. But it doesn't work with threads, as they have no explicit yield or switch. On Wed, Apr 4, 2012 at 11:07 PM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
Same applies here. But you propagate return value/error right? So you can't say "frames are not connected". They aren't from the interpreter point of view. But they are logically connected. So for example: def a(): yield b() def b(): yield If `a().with_timeout(0.1)` is interrupted when it's waiting for value of `b()`, will `b()` continue it's execution?
For us, having 'f_in_finally' somehow propagated would be completely useless.
I hope I can convince you with this email :)
I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame.
That was original intention. But it requires stack traversing. Andrew proposed to propagate this flag, which is another point of view on the same thing (not sure which one to pick though) -- Paul
On 2012-04-04, at 4:43 PM, Paul Colomiets wrote:
Interesting. I decided to go with a simple coroutine objects with a '_caller' pointer to maintain the stack virtually.
Are you using that particular framework (zorro)? Or some modification of it that uses generators too?
OK.
OK, we're on the same page. '''"frames are not connected" from the interpreter point of view''', that essentially means that 'f_in_finally' will always be a flag related only to its own frame, right? Any 'propagation' of this flag is the responsibility of framework developers.
Well, in our framework, if a() is getting aborted after it's scheduled b(), but before it received the result of b(), we interrupt both of them (and those that b() might had called).
Again, if coroutines' frames aren't connected on the interpreter level (it's the responsibility of a framework), about what exact propagation are you and Andrew talking (in the sole context of the patch to cpython)? - Yury
Hi Yury, On Thu, Apr 5, 2012 at 12:24 AM, Yury Selivanov <yselivanov.ml@gmail.com> wrote:
It doesn't matter. IIRC, that was to draw a tree of calls starting with roots. But that's irrelevant to the topic of discussion.
Currently we experimenting with greenlets and zorro. My description of yield-based coroutines from earlier project (unfortunately non-public one).
Yes, that's ok for me.
Exactly! And you don't want them to be interrupted in the case `a()` rewritten as: def a(): try: pass finally: yield b() Same with threads and greenlets.
For threads and greenlets flag can be implicitly propagated, and for yield-based coroutines f_in_finally can be made writable, so you can propagate it in you own scheduler Not sure It's good idea, just describing it. -- Paul
On 2012-04-04, at 2:44 PM, Paul Colomiets wrote:
BTW, for instance, in our framework each coroutine is a special object that wraps generator/plain function. It controls everything that the underlying generator/function yields/returns. But the actual execution, propagation of returned values and raised errors is the scheduler's job. So when you are yielding a coroutine from another coroutine, frames are not even connected to each other, since the actual execution of the callee will be performed by the scheduler. It's not like a regular python call. For us, having 'f_in_finally' somehow propagated would be completely useless. I think even if it's decided to implement just your proposal, I feel that 'f_in_finally' should indicate the state of only its *own* frame. - Yury
Paul Colomiets wrote:
It wouldn't be in each frame -- probably it would just be a global hook that gets called whenever a finally-counter anywhere gets decremented from 1 to 0. It would be passed the relevant frame so it could decide what to do from there. I don't think it would have much performance impact, since it would only get triggered by exiting a finally block. Nothing would need to happen per bytecode or anything like that. -- Greg
Hi Greg, On Thu, Apr 5, 2012 at 1:18 AM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
It's similar to sys.settrace() except it only executed when finally counter decremented to 0, right? Flag `f_in_finally` is still there, right? It solves my use case well. Yury, is it ok if l'll start a PEP with this idea, and when it will have some support (or be rejected), you'll come up with thread interruption proposal? -- Paul
On 2012-04-04, at 6:45 PM, Paul Colomiets wrote:
Yes, please keep it. With your current proposal, it's the only way to see if it is safe to interrupt coroutine right now, or we have to wait until the callback gets called.
Sure, go ahead. If I'm lucky enough to come up with a better proposal I promise to shout about it loud ;) - Yury
Paul Colomiets wrote:
I don't think a frame flag on its own is quite enough. You don't just want to prevent interruptions while in a finally block, you want to defer them until the finally counter gets back to zero. Making the interrupter sleep and try again in that situation is rather ugly. So perhaps there could also be a callback that gets invoked when the counter goes down to zero. -- Greg
On Mon, Apr 2, 2012 at 3:28 PM, Yury Selivanov <yselivanov.ml@gmail.com>wrote:
A context manager doesn't solve this interruption "race condition" issue anyways. If the __enter__ method is interrupted it won't have returned a context and thus __exit__ will never be called. -gps
On 2012-04-02, at 7:10 PM, Gregory P. Smith wrote:
If the __enter__ method is interrupted it won't have returned a context and thus __exit__ will never be called.
To address that Paul proposed to make __enter__ non-interruptable as well. - Yury
participants (5)
-
Andrew Svetlov -
Greg Ewing -
Gregory P. Smith -
Paul Colomiets -
Yury Selivanov