[Python-ideas] The async API of the future: yield-from

[Guido van Rossum]

On Sat, Oct 20, 2012 at 4:21 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Simply having the scheduler know where all the tasks are
> will not prevent deadlocks. It might make it possible for the
> scheduler to *detect* deadlocks, but you still have to do
> something about them.
>
> Having said that, I'm thinking about writing a more elaborate
> version of my scheduler that does keep track of which queue a
> task is waiting on, mainly so that tasks can be cancelled
> cleanly.

In NDB, I have a feature that detects most deadlocks -- the Future
class keeps track of all incomplete instances, and it can dump this
list at request. Futures also keep some information about where and
for what purpose they were created. Finally, to tie it all together,
there's code that detects that you're waiting for something to happen
but the event loop is out of things to do (i.e. no pending RPCs, no
"call later" callbacks left -- hence, no progress can possibly be
made).

This feature has caught mostly bugs in NDB itself -- because NDB is
primarily a database API, regular NDB users don't normally write code
that is likely to deadlock. But in the wider Python 3 world, where
regular users would be writing (presumably buggy) protocol
implementations and their own servers and clients, I suspect debugging
features can make and break a system like this.

-- 
--Guido van Rossum (python.org/~guido)