[Python-ideas] Thoughts on the future of concurrency in Python: PEP 3148

[Nick Coghlan]

On 17 Mar 2013 11:35, "Steven Hazel" <sah at awesame.org> wrote:
>
> Hi,
>
> I'm one of the authors of monocle (https://github.com/saucelabs/monocle),
an async Python programming framework with a blocking look-alike syntax
based on generators. We've been using monocle at my startup Sauce Labs, and
I used its predecessor on a small team at BitTorrent. I think it's met with
some amazing success vs. similar projects using threads. In about five
years of working with teams with some members who haven't always understood
monocle very well, we've never written a deadlock, for example.
>
> I've been keeping up with PEPs and skimming python-ideas discussions
about concurrency-related features in Python, and feeling unsure about how
best to jump into the conversation. Today it occurred to me that sooner is
better and I should just get all my various opinions out there in case
they're helpful.
>
> First of all, I want to reiterate: while I have mainly criticism to
offer, it is only intended to be helpful. I don't have a lot of time to
devote to this myself, so I'm trying to do what I can to be useful to those
who do.
>
> I think I'll take this one PEP at a time, and start with the easiest one.
>
> PEP 3148 - Futures

This PEP isn't new, it was released in Python 3.2, and is available for 2.x
as a backport on PyPI.

Cheers,
Nick.

>
> First, I like the idea of this PEP. I think it'll improve
interoperability between async frameworks in Python.
>
> I believe the name "future" is a poor choice. After many years of
explaining this concept to programmers, I've found that its various
abstract-conceptual names do a lot to confuse people. Twisted's "deferred"
is probably the worst offender here, but both "future" and "promise" also
make this very simple construct sound complicated, and it's a crime.
>
> There are two ways I've found to explain this that seem to keep people
from getting confused:
>
> The first is, call it a "callback registry". Many programmers are
familiar with the idea of passing callbacks into a function. You just
explain that here, instead of taking the callbacks as parameters, we return
a callback registry object and you can add your callbacks there. (If the
operation is already done they'll just get called when you add them.)
>
> The second is a variant on that: say that this function, instead of
taking a callback as a parameter, returns its callback. That sounds
useless, but fortunately the callback it returns is a callable object, and
it also has an "add" method that lets you add functions that it will call
when called back. So it's a callback that you can manipulate to do what you
want. (This is what we do in monocle:
https://github.com/saucelabs/monocle/blob/master/monocle/callback.py)
>
> These approaches, by focusing on the simple and familiar idea of
callbacks, do a lot to eliminate confusing ideas about communicating with
the future of the process.
>
> Finally, I'm not sure I get the point of the Executor class in PEP 3148.
It seems to me that any real implementation of that class needs a
call_later method for scheduling operations, and a way of connecting up to
event-based IO APIs. I don't really understand what Executor would be good
for.
>
> Thanks for reading.
>
> --
> Steven Hazel
> Sauce Labs Cofounder and VP Product
> http://saucelabs.com/
> work: sah at saucelabs.com
> other: sah at awesame.org
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130317/7ef9ff6b/attachment.html>