[Python-Dev] Pythonic concurrency

Bruce Eckel BruceEckel-Python3234 at mailblocks.com
Thu Oct 6 22:06:37 CEST 2005

This does look quite fascinating, and I know there's a lot of really
interesting work going on at the BBC now -- looks like some really
pioneering stuff going on with respect to TV show distribution over
the internet, new compression formats, etc.

So yes indeed, this is quite high on my list to research. Looks like
people there have been doing some interesting work.

Right now I'm just trying to cast a net, so that people can put in
ideas, for when the Java book is done and I can spend more time on it.

Thursday, October 6, 2005, 1:54:56 PM, Michael Sparks wrote:

> Hi Bruce,

> On Thursday 06 October 2005 18:12, Bruce Eckel wrote:
>> Although I hope our conversation isn't done, as he suggests!
> ...
>> At some point when more ideas have been thrown about (and TIJ4 is
>> done) I hope to summarize what we've talked about in an article.

> I don't know if you saw my previous post[1] to python-dev on this topic, but
> Kamaelia is specifically aimed at making concurrency simple and easy to use.
> Initially we were focussed on using scheduled generators for co-operative
> CSP-style (but with buffers) concurrency.
>    [1] http://tinyurl.com/dfnah, http://tinyurl.com/e4jfq

> We've tested the system so far on 2 relatively inexperienced programmers
> (as well as experienced, but the more interesting group is novices). The one
> who hadn't done much programming at all (a little bit of VB, pre-university)
> actually fared better IMO. This is probably because concurrency became
> part of his standard toolbox of approaches.

> I've placed the slides I've produced for Euro OSCON on Kamaelia here:
>    * http://cerenity.org/KamaeliaEuroOSCON2005.pdf

> The corrected URL for the whitepaper based on work now 6 months old (we've
> come quite a way since then!) is here:
>    * http://www.bbc.co.uk/rd/pubs/whp/whp113.shtml

> Consider a simple server for sending text (generated by a user typing into the
> server) to multiple clients connecting to a server. This is a naturally
> concurrent problem in various ways (user interaction, splitting, listening
> for connections, serving connections, etc). Why is that interesting to us?
> It's effectively a microcosm of how subtitling works. (I work at the BBC)

> In Kamaelia this looks like this:

> === start ===
> class ConsoleReader(threadedcomponent):
>    def run(self):
>       while 1:
>          line = raw_input(">>> ")
>          line = line + "\n"
>          self.outqueues["outbox"].put(line)

> Backplane("subtitles").activate()
> pipeline(
>     ConsoleReader(),
>     publishTo("subtitles"),
> ).activate()
> def subtitles_protocol():
>     return subscribeTo("subtitles")

> SimpleServer(subtitles_protocol, 5000).run()
> === end ===

> The ConsoleReader is threaded to allow the use of the naive way of
> reading from the input, whereas the server, backplane (a named splitter
> component in practice), pipelines, publishing, subscribing, splitting,
> etc are all single threaded co-operative concurrency.

> A possible client for this text service might be:

> pipeline(
>     TCPClient("subtitles.rd.bbc.co.uk", 5000),
>     Ticker(),
> ).run()

> (Though that would be a bit bare, even if it does use pygame :)

> The entire system is based around communicating generators, but we also
> have threads for blocking operations. (Though the entire network subsystem
> is non-blocking)

> What I'd be interested in, is hearing how our system doesn't match with
> the goals of the hypothetical concurrency system you'd like to see (if it
> doesn't). The main reason I'm interested in hearing this, is because the
> goals you listed are ones we want to achieve. If you don't think our system
> matches it (we don't have process migration as yet, so that's one area)
> I'd be interested in hearing what areas you think are deficient.

> However, the way we're beginning to refer to the project is to refer to
> just the component aspect rather than concurrency - for one simple
> reason - we're getting to stage where we can ignore /most/ concurrency
> issues(not all).

> If you have any time for feedback, it'd be appreciated. If you don't I hope
> it's useful food for thought! 

> Best Regards,

> Michael

Bruce Eckel    http://www.BruceEckel.com   mailto:BruceEckel-Python3234 at mailblocks.com
Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e"
Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel
Subscribe to my newsletter:
My schedule can be found at:

More information about the Python-Dev mailing list