[Python-ideas] The async API of the future: Twisted and Deferreds

[Itamar Turner-Trauring]

(Sorry if this doesn't end up in the right thread in mail clients; I've
been reading this through a web UI and only just formally subscribed so
can't reply directly to the correct email.)

Code that uses generators is indeed often easier to read... but the problem
is that this isn't just a difference in syntax, it has a significant
semantic impact. Specifically, requiring yield means that you're
re-introducing context switching. In inlineCallbacks, or coroutines, or any
system that use yield as in your example above, arbitrary code may run
during the context switch, and who knows what happened to the state of the
world in the interim. True, it's an explicit context switch, unlike
threading where it can happen at any point, but it's still a context
switch, and it still increases the chance of race conditions and all the
other problems threading has. (If you're omitting yield it's even worse,
since you can't even tell anymore where the context switches are
happening.) Superficially such code is simpler (and in some cases I'm happy
to use inlineCallbacks, in particular in unit tests), but much the same way
threaded code is "simpler". If you're not very very careful, it'll work 99
times and break mysteriously the 100th.

For example, consider the following code; silly, but buggy due to the
context switch in yield allowing race conditions if any other code modifies
counter.value while getResult() is waiting for a result.

   def addToCounter():
        counter.value = counter.value + (yield getResult())

In a Deferred callback, on the other hand, you know the only things that
are going to run are functions you call. In so far as it's possible, what
happens is under control of one function only. Less pretty, but no
potential race conditions:

    def add(result):
        counter.value = counter.value + result
    getResult().addCallback(add)

That being said, perhaps some changes to Python syntax could solve this;
Allen Short (
http://washort.twistedmatrix.com/2012/10/coroutines-reduce-readability.html)
claims to have a proposal, hopefully he'll post it soon.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20121013/e931fbc9/attachment.html>