[Python-ideas] Fwd: Concurrent safety?

[Stephen J. Turnbull]

Mike Meyer writes:

 > The cpu manufacturers have switched to improving performance by adding
 > more cores instead of cranking clock speeds. As time goes by, more and
 > more programmers are going to want to leverage that in serious
 > ways.

Please, everybody is aware of that.  Anybody against improved support
for concurrency is probably in favor of criminalizing motherhood and
apple pie, too.  What you need to justify is the apparently expensive
approach you propose, and you need to resolve the apparent
contradiction between that expense and the basic argument for threads
which is precisely how expensive they *aren't*.

 > I've identified the problem I want to solve: I want to make
 > concurrent use of python objects "safe by default",

But that's not what you've proposed, AIUI.  You've proposed making
concurrent use *safer*, but not yet *safe*.  That's quite different
from the analogy with automatic memory management, where the
programmer can't do anything dangerous with pointers (because they
can't do anything at all).  The analogous model for concurrency is
processes, it seems to me.  (I don't have a desperate need for high-
performance concurrency, so I take no position on processes + message
passing vs. threads + shared resources.)

 > so that doing unsafe things causes the programmer to have to do
 > something explicit about making things safe.

This is un-Pythonic, IMO[1].  Python generally permits dangerous (and
even ugly) things when done by "consenting adults", on the theory that
the programmer knows more about her problem than Python does.  It
seems to me that a more Pythonic approach to this would be to provide
something like STM as a metaclass, mixin class, or decorator.  (Don't
ask me how.)

 > I believe this can be done at the mutation points (again, clojure
 > shows that it can be done).

But clojure is a Lisp-derived language.  Lisp was designed as a pure
functional language (although AFAIK it pretty much immediately
acquired "set"), and a very large number of Lisp algorithms are
designed around conses which are (like Python tuples) basically
immutable (yes, I know about setcar and setcdr, but use of those
functions is generally considered a bug).  Whether that orientation
toward immutable objects continues in Clojure I don't know, but if it
does, the problem of designing a "write barrier" for mutations may be
(a) simpler and (b) have less performance impact than the analogous
task applied to Python.

While Python-the-language does have some immutable objects like tuples
and strings, it's really kind of hard to avoid use of containers like
lists and dictionaries and classes with mutable objects.


Footnotes: 
[1]  But I've been clobbered for expressing my opinion on Pythonicity
in the past, so don't put too much weight on that.<wink/>