[Python-ideas] Thoughts on the future of concurrency in Python: PEP 3148

[Guido van Rossum]

Hi Steven!

On Sun, Mar 17, 2013 at 11:34 AM, Steven Hazel <sah at awesame.org> wrote:
> Hi,
>
> I'm one of the authors of monocle (https://github.com/saucelabs/monocle), an
> async Python programming framework with a blocking look-alike syntax based
> on generators. We've been using monocle at my startup Sauce Labs, and I used
> its predecessor on a small team at BitTorrent. I think it's met with some
> amazing success vs. similar projects using threads. In about five years of
> working with teams with some members who haven't always understood monocle
> very well, we've never written a deadlock, for example.
>
> I've been keeping up with PEPs and skimming python-ideas discussions about
> concurrency-related features in Python, and feeling unsure about how best to
> jump into the conversation. Today it occurred to me that sooner is better
> and I should just get all my various opinions out there in case they're
> helpful.
>
> First of all, I want to reiterate: while I have mainly criticism to offer,
> it is only intended to be helpful. I don't have a lot of time to devote to
> this myself, so I'm trying to do what I can to be useful to those who do.
>
> I think I'll take this one PEP at a time, and start with the easiest one.
>
> PEP 3148 - Futures
>
> First, I like the idea of this PEP. I think it'll improve interoperability
> between async frameworks in Python.
>
> I believe the name "future" is a poor choice. After many years of explaining
> this concept to programmers, I've found that its various abstract-conceptual
> names do a lot to confuse people. Twisted's "deferred" is probably the worst
> offender here, but both "future" and "promise" also make this very simple
> construct sound complicated, and it's a crime.
>
> There are two ways I've found to explain this that seem to keep people from
> getting confused:
>
> The first is, call it a "callback registry". Many programmers are familiar
> with the idea of passing callbacks into a function. You just explain that
> here, instead of taking the callbacks as parameters, we return a callback
> registry object and you can add your callbacks there. (If the operation is
> already done they'll just get called when you add them.)
>
> The second is a variant on that: say that this function, instead of taking a
> callback as a parameter, returns its callback. That sounds useless, but
> fortunately the callback it returns is a callable object, and it also has an
> "add" method that lets you add functions that it will call when called back.
> So it's a callback that you can manipulate to do what you want. (This is
> what we do in monocle:
> https://github.com/saucelabs/monocle/blob/master/monocle/callback.py)
>
> These approaches, by focusing on the simple and familiar idea of callbacks,
> do a lot to eliminate confusing ideas about communicating with the future of
> the process.
>
> Finally, I'm not sure I get the point of the Executor class in PEP 3148. It
> seems to me that any real implementation of that class needs a call_later
> method for scheduling operations, and a way of connecting up to event-based
> IO APIs. I don't really understand what Executor would be good for.
>
> Thanks for reading.
>
> --
> Steven Hazel
> Sauce Labs Cofounder and VP Product
> http://saucelabs.com/
> work: sah at saucelabs.com
> other: sah at awesame.org
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> http://mail.python.org/mailman/listinfo/python-ideas
>



--
--Guido van Rossum (python.org/~guido)