[Python-ideas] Async API

[Guido van Rossum]

On Fri, Oct 26, 2012 at 7:12 AM, Itamar Turner-Trauring
<itamar at futurefoundries.com> wrote:
>
>
> On Wed, Oct 24, 2012 at 7:43 PM, Guido van Rossum <guido at python.org> wrote:
>>
>>
>> > I don't know, I hope someone with an expertise in Twisted can tell us.
>> >
>> > But I would imagine that they don't have this particular problem, as it
>> > should be related only to coroutines and schedulers that run them.  I.e.
>> > it's a problem when you run some code and may interrupt it.  And you
>> > can't
>> > interrupt a plain python code that uses callbacks without yields and
>> > greenlets.
>>
>> Well, but in the Twisted world, if a cleanup callback requires more
>> blocking calls, it has to spawn more deferred callbacks. So I think
>> they *do* have the problem, unless they don't have a way at all to
>> constrain the total running time of an action involving cascading
>> callbacks. Also, they have inlineCallbacks which does use yield.
>
>
> Deferreds don't do anything to prevent blocking. They're just a nice
> abstraction for callbacks. And yes, if you call 1000 functions that do lots
> of CPU in a row, that will keep other stuff from happening.
>
> However, consider how a timeout works: the event loop notices enough time
> has passed, and so calls some code that tells the Deferred to cancel its
> operation. So you're *not* adding the cancellation operations to the stack
> of the original operation, you're starting from the event loop. And so
> timeouts are just normal event loop world, where you need to be careful not
> to do to much CPU-intensive processing in any given call, and you can't call
> blocking system calls (except using a thread).
>
> Of course, you can't timeout a function that's just looping using CPU, or a
> blocking system call, and so code needs to be structured to deal with this,
> but that's a different issue.

So, basically, it's just "after T seconds you get this second callback
and it's up to you to deal with it"? I guess the timeout callback can
inspect the state of the operation, and cancel any pending operations?

Do you have a way to translate timeouts into exceptions in
inlineCallbacks? If so, how is that working out?

-- 
--Guido van Rossum (python.org/~guido)