[Python-ideas] Protecting finally clauses of interruptions
yselivanov.ml at gmail.com
Wed Apr 4 03:23:34 CEST 2012
On 2012-04-03, at 3:22 PM, Paul Colomiets wrote:
> (Although, I don't know how `yield from` changes working with
> yield-based coroutines, may be it's behavior is quite different)
> For greenlets situation is a bit different, as Python knows the
> stack there, but you still need to traverse it (or as Andrew
> mentioned, you can just propagate flag).
Why traverse? Why propagate? As I explained in my previous posts
here, you need to protect only the top-stack coroutines in the
timeouts or trampoline execution queues. You should illustrate
your logic with a more clear example - say three or four coroutines
that call each other + with a glimpse of how your trampoline works.
But I'm not sure that is really necessary.
>> - no mechanism for interrupting a running thread.
> Yes. That was intentionally to have greater chance to success.
> Interruption may be separate proposal.
>> In almost any coroutine library you will have a thread pool,
>> and sometimes you need a way to interrupt workers.
> Which one? I know that twisted uses thread pool to handle:
Besides Twisted? eventlet; gevent will have them in 1.0, etc.
> 1. DNS (which is just silly)
> 2. Few other protocols which doesn't have asynchronous
> libraries (which should be fixed)
> The whole intention of using coroutine library is to not to
> have thread pool. Could you describe your use case
> with more details?
Well, our company has been using coroutines for like 2.5 years
now (the framework in not yet opensourced). And in our practice
threadpool is really handy, as it allows you to:
- use non-asyncronous libraries, which you don't want to
monkeypatch with greensockets (or even unable to mokeypatch)
- wrap some functions that are usually very fast, but once in
a while may take some time. And sometimes you don't want to
offload them to a separate process
- and yes, do DNS lookups if you don't have a compiled cpython
extension that wraps c-ares or something alike.
Please let's avoid shifting further discussion to proving or
disproving the necessity of threadpools. They are being actively
used and there is a demand for (more or less) graceful threads
interruption or abortion.
>> So it's not enough even for coroutines.
> Very subjective, and doesn't match my expectations.
As I said -- we've been working with coroutines (combined
generators + greenlets) for a few years, and apparently have
different experience, opinions and expectations from what you
have. And I suppose developers and users of eventlet, gevent,
twisted (@inlineCallbacks) and other libraries have their own
opinions and ideas too. Not to mention, it would be interesting
to hear from PyPy, Juthon and IronPython teams. It also seems
that neither of us have enough experience working with
'yield from' style coroutines.
Please write a PEP and we'll continue discussion from that
point. Hopefully, it will get more attention than this thread.
More information about the Python-ideas