[Python-Dev] Pythonic concurrency

Josiah Carlson jcarlson at uci.edu
Fri Oct 7 00:15:07 CEST 2005

Michael Sparks <ms at cerenity.org> wrote:
> What I'd be interested in, is hearing how our system doesn't match with
> the goals of the hypothetical concurrency system you'd like to see (if it
> doesn't). The main reason I'm interested in hearing this, is because the
> goals you listed are ones we want to achieve. If you don't think our system
> matches it (we don't have process migration as yet, so that's one area)
> I'd be interested in hearing what areas you think are deficient.

I've not used the system you have worked on, so perhaps this is easy,
but the vast majority of concurrency issues can be described as fitting
into one or more of the following task distribution categories.

1. one to many (one producer, many consumers) without duplication (no
consumer has the same data, essentially a distributed queue)
2. one to many (one producer, many consumers) with duplication (the
producer broadcasts to all consumers)
3. many to one (many producers, one consumer)
4. many to many (many producers, many consumers) without duplication (no
consumer has the same data, essentially a distributed queue)
5. many to many (many producers, many consumers) with duplication (all
producers broadcast to all consumers)
6. one to one without duplication

MPI, for example, handles all the above cases with minor work, and
tuple space systems such as Linda can support all of the above with a
bit of work in cases 2 and 5.

If Kamaelia is able to handle all of the above mechanisms in both a
blocking and non-blocking fashion, then I would guess it has the basic
requirements for most concurrent applications.  If, however, it is not
able to easily handle all of the above mechanisms, or has issues with
blocking and/or non-blocking semantics on the producer and/or consumer
end, then it is likely that it will have difficulty gaining traction in
certain applications where the unsupported mechanism is common and/or

One nice thing about the message queue style (which it seems as though
Kamaelia implements) is that it guarantees that a listener won't recieve
the same message twice when broadcasting a message to multiple listeners
(case 2 and 5 above) - something that is a bit more difficult to
guarantee in a tuple space scenario, but which is still possible (which
spurns me to add it into my tuple space implementation before it is
released). Another nice thing is that subscriptions to a queue seem to
be persistant in Kamaelia, which I should also implement.

 - Josiah

More information about the Python-Dev mailing list