[Python-Dev] Pythonic concurrency

[Bruce Eckel]

I'd like to restart this discussion; I didn't mean to put forth active
objects as "the" solution, only that it seems to be one of the better,
more OO solutions that I've seen so far.

What I'd really like to figure out is the "pythonic" solution for
concurrency. Guido and I got as far as agreeing that it wasn't
threads.

Here are my own criteria for what such a solution would look like:

1) It works by default, so that novices can use it without falling
into the deep well of threading. That is, a program that you write
using threading is broken by default, and the tool you have to fix it
is "inspection." I want something that allows me to say "this is a
task. Go." and have it work without the python programmer having to
study and understand several tomes on the subject.

2) Tasks can be automatically distributed among processors, so it
solves the problems of (a) making python run faster (b) how to utilize
multiprocessor systems.

3) Tasks are cheap enough that I can make thousands of them, to solve
modeling problems (in which I also lump games). This is really a
solution to a cerain type of program complexity -- if I can just
assign a task to each logical modeling unit, it makes such a system
much easier to program.

4) Tasks are "self-guarding," so they prevent other tasks from
interfering with them. The only way tasks can communicate with each
other is through some kind of formal mechanism (something queue-ish,
I'd imagine).

5) Deadlock is prevented by default. I suspect livelock could still
happen; I don't know if it's possible to eliminate that.

6) It's natural to make an object that is actor-ish. That is, this
concurrency approach works intuitively with objects.

7) Complexity should be eliminated as much as possible. If it requires
greater limitations on what you can do in exchange for a clear,
simple, and safe programming model, that sounds pythonic to me. The
way I see it, if we can't easily use tasks without getting into
trouble, people won't use them. But if we have a model that allows
people to (for example) make easy use of multiple processors, they
will use that approach and the (possible) extra overhead that you pay
for the simplicity will be absorbed by the extra CPUs.

8) It should not exclude the possibility of mobile tasks/active
objects, ideally with something relatively straightforward such as
Linda-style tuple spaces.

One thing that occurs to me is that a number of items on this wish
list may conflict with each other, which may require a different way
of thinking about the problem. For example, it may require two
approaches: for "ordinary" non-OO tasks, a functional programming
approach ala Erlang, in combination with an actor approach for
objects.

Bruce Eckel    http://www.BruceEckel.com   mailto:BruceEckel-Python3234 at mailblocks.com
Contains electronic books: "Thinking in Java 3e" & "Thinking in C++ 2e"
Web log: http://www.artima.com/weblogs/index.jsp?blogger=beckel