[Python-ideas] Async API

[Steve Dower]

>On 2012-10-24, at 8:00 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>> On 2012-10-24, at 7:43 PM, Guido van Rossum <guido at python.org> wrote:
>>> On Wed, Oct 24, 2012 at 4:26 PM, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>>>> On 2012-10-24, at 7:12 PM, Guido van Rossum <guido at python.org> wrote:
>>>>> Ok, I can understand. But still, this is a problem with timeouts in 
>>>>> general, not just with timeouts in a yield-based environment. How 
>>>>> does e.g. Twisted deal with this?
>>> 
>>>> I don't know, I hope someone with an expertise in Twisted can tell us.
>>>> 
>>>> But I would imagine that they don't have this particular problem, as 
>>>> it should be related only to coroutines and schedulers that run them.  I.e.
>>>> it's a problem when you run some code and may interrupt it.  And you 
>>>> can't interrupt a plain python code that uses callbacks without 
>>>> yields and greenlets.
>>> 
>>> Well, but in the Twisted world, if a cleanup callback requires more 
>>> blocking calls, it has to spawn more deferred callbacks. So I think 
>>> they *do* have the problem, unless they don't have a way at all to 
>>> constrain the total running time of an action involving cascading 
>>> callbacks. Also, they have inlineCallbacks which does use yield.
>> 
>> Right.
>> 
>> I was under impression that you don't just use 'finally' stmt but 
>> rather setup a Deferred with a cleanup callback.  Anyways, I'm now 
>> curious enough so I'll take a look...
>
>Well, that wasn't too hard to find:
>
>Timeouts:
>http://stackoverflow.com/questions/221745/is-it-possible-to-set-a-timeout-on-a-socket-in-twisted


Maybe our approach to timeouts should be based on running two tasks in parallel, where the second delays for the timeout period and then cancels the first (I believe this is what they're doing in Twisted). My vision for cancellation involves the worker task polling (or whatever is appropriate for low-level tasks), rather than an exception being forced in by the scheduler, so this avoids the finally issue - it's too late to cancel the task at that point.

It also strengthens the case for including a cancellation protocol, which I was keen on anyway.


Cheers,
Steve