[Python-ideas] Protecting finally clauses of interruptions

Yury Selivanov yselivanov.ml at gmail.com
Wed Apr 4 03:23:34 CEST 2012


On 2012-04-03, at 3:22 PM, Paul Colomiets wrote:
> (Although, I don't know how `yield from` changes working with
> yield-based coroutines, may be it's behavior is quite different)
> 
> For greenlets situation is a bit different, as Python knows the
> stack there, but you still need to traverse it (or as Andrew
> mentioned, you can just propagate flag).

Why traverse?  Why propagate?  As I explained in my previous posts 
here, you need to protect only the top-stack coroutines in the 
timeouts or trampoline execution queues.  You should illustrate
your logic with a more clear example - say three or four coroutines
that call each other + with a glimpse of how your trampoline works.
But I'm not sure that is really necessary.

>> - no mechanism for interrupting a running thread.
> 
> Yes. That was intentionally to have greater chance to success.
> Interruption may be separate proposal.
> 
>> In almost any coroutine library you will have a thread pool,
>> and sometimes you need a way to interrupt workers.
> 
> Which one? I know that twisted uses thread pool to handle:

Besides Twisted?  eventlet; gevent will have them in 1.0, etc.

> 1. DNS (which is just silly)
> 2. Few other protocols which doesn't have asynchronous
> libraries (which should be fixed)
> 
> The whole intention of using coroutine library is to not to
> have thread pool. Could you describe your use case
> with more details?

Well, our company has been using coroutines for like 2.5 years
now (the framework in not yet opensourced).  And in our practice
threadpool is really handy, as it allows you to:

- use non-asyncronous libraries, which you don't want to
monkeypatch with greensockets (or even unable to mokeypatch)

- wrap some functions that are usually very fast, but once in
a while may take some time.  And sometimes you don't want to
offload them to a separate process

- and yes, do DNS lookups if you don't have a compiled cpython
extension that wraps c-ares or something alike.

Please let's avoid shifting further discussion to proving or 
disproving the necessity of threadpools.  They are being actively 
used and there is a demand for (more or less) graceful threads 
interruption or abortion.

>> So it's not enough even for coroutines.
>> 
> 
> Very subjective, and doesn't match my expectations.


As I said -- we've been working with coroutines (combined
generators + greenlets) for a few years, and apparently have
different experience, opinions and expectations from what you
have.  And I suppose developers and users of eventlet, gevent,
twisted (@inlineCallbacks) and other libraries have their own 
opinions and ideas too.  Not to mention, it would be interesting
to hear from PyPy, Juthon and IronPython teams.  It also seems 
that neither of us have enough experience working with 
'yield from' style coroutines.

Please write a PEP and we'll continue discussion from that 
point.  Hopefully, it will get more attention than this thread.

-
Yury



More information about the Python-ideas mailing list