[Python-ideas] Async API

[Yury Selivanov]

On 2012-10-24, at 7:43 PM, Guido van Rossum <guido at python.org> wrote:
> On Wed, Oct 24, 2012 at 4:26 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>> On 2012-10-24, at 7:12 PM, Guido van Rossum <guido at python.org> wrote:
>>> Ok, I can understand. But still, this is a problem with timeouts in
>>> general, not just with timeouts in a yield-based environment. How does
>>> e.g. Twisted deal with this?
> 
>> I don't know, I hope someone with an expertise in Twisted can tell us.
>> 
>> But I would imagine that they don't have this particular problem, as it
>> should be related only to coroutines and schedulers that run them.  I.e.
>> it's a problem when you run some code and may interrupt it.  And you can't
>> interrupt a plain python code that uses callbacks without yields and
>> greenlets.
> 
> Well, but in the Twisted world, if a cleanup callback requires more
> blocking calls, it has to spawn more deferred callbacks. So I think
> they *do* have the problem, unless they don't have a way at all to
> constrain the total running time of an action involving cascading
> callbacks. Also, they have inlineCallbacks which does use yield.

Right.

I was under impression that you don't just use 'finally' stmt but rather
setup a Deferred with a cleanup callback.  Anyways, I'm now curious enough
so I'll take a look...

> Note that in a world with only blocking calls this *can* be a problem
> (despite your repeated claims that it's not a problem there) -- a
> common approach to giving operations a timeout is sending it a SIGTERM
> (which you can easily call with a signal handler in Python) when the
> deadline is over, then sending it more SIGTERM signals every few
> seconds until it dies, and sending SIGKILL (which can't be caught) if
> it takes too long to die.

Yes, you're right.  I guess I've just never seen anybody trying to protect
their 'finally' statements from being interrupted with a signal.

Whereas with coroutines we needed to protect lots of them, as otherwise
we had many and many bugs with unclosed database connections etc.

So 'protect_finally' is going to be a very common thing to use.

-
Yury