[Python-ideas] Thoughts on the future of concurrency in Python: PEP 3148
sah at awesame.org
Sun Mar 17 23:14:29 CET 2013
On Sunday, March 17, 2013 2:06:48 PM UTC-7, Guido van Rossum wrote:
> As Nick pointed out, this water is already under the 3.2 bridge. Too
> late to change now.
Oops, didn't realize it was already out. In that case, please take this as
a suggestion that the docs might benefit from an explanation like "A Future
is basically just a callback registry. Rather than taking callbacks as
parameters, functions can return a Future, where callbacks can be
registered with Future.add_done_callback."
> I actually want to de-emphasize the fact that Futures hold callbacks.
> I want people to think them as magic "wait points" that combine with
> "yield from" to return a pseudo-synchronous result. In fact, I want to
> altogether de-emphasize that async I/O is done under the hood using
> callbacks. Only implementers of async frameworks should need to know
> that, for the most part.
It's true that when you're using a generator-based async framework, you can
and should think of a Future as a "thing you can wait on" most of the time.
My experience with monocle though is that it is helpful rather than harmful
to reveal that they're about callbacks. In the early days of monocle, we
were using Deferreds, which a lot of new monocle users didn't really
understand, and people tended to get very confused about what kinds of
things a Deferred could possibly be doing. Explaining things in terms of
callbacks was helpful in getting people to understand monocle.
I don't think you can really abstract the idea of callbacks away from
Futures without making them more mystifying. Callbacks are not an
implementation detail of Futures, they're essential to what Futures do, so
essential that I think CallbackRegistry is pretty good alternative name,
and in monocle we actually called the class Callback. I sometimes explain
that a monocle oroutine is "yielding on a callback" — meaning it's waiting
now and it'll resume when called back. The name helps explain what's
happening, even for an user of the framework.
Hm. I find the idea of callbacks far from simple nor familiar.
Many programmers have used callbacks in other contexts. But, even when they
haven't, "you pass in a function that will get called when the operation is
done" is an idea I've watched many people grasp immediately the first time
they saw it. In contrast, Futures, which are almost the same idea, are
often viewed as sophisticated black magic, and I've more than once heard
them explained in terms of *time travel*.
> It's a threadpool.
Oh, I see. Well, for what it's worth, including this next to Futures
confused me. I kind of implied to me when I read the PEP that Executors are
somehow necessary to using Futures, when in fact they're just one of many
contexts in which a Future might be a good API.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-ideas