[Python-Dev] Pythonic concurrency
shane at hathawaymix.org
Fri Oct 7 20:42:02 CEST 2005
Bruce Eckel wrote:
> But. I do happen to have contact with a lot of people who are at the
> forefront of the threading world, and *none* of them (many of whom
> have written the concurrency libraries for Java 5, for example) ever
> imply that threading is easy. In fact, they generally go out of their
> way to say that it's insanely difficult.
What's insanely difficult is really locking, and locking is driven by
concurrency in general, not just threads. It's hard to reason about
locks. There are only general rules about how to apply locking
correctly, efficiently, and without deadlocks. Personally, to be
absolutely certain I've applied locks correctly, I have to think for
hours. Even then, it's hard to express my conclusions, so it's hard to
be sure future maintainers will keep the locking correct.
Java uses locks very liberally, which is to be expected of a language
that provides locking using a keyword. This forces Java programmers to
deal with the burden of locking everywhere. It also forces the
developers of the language and its core libraries to make locking
extremely fast yet safe. Java threads would be easy if there wasn't so
much locking going on.
Zope, OTOH, is far more conservative with locks. There is some code
that dispatches HTTP requests to a worker thread, and other code that
reads and writes an object database, but most Zope code isn't aware of
concurrency. Thus locking is hardly an issue in Zope, and as a result,
threading is quite easy in Zope.
Recently, I've been simulating high concurrency on a PostgreSQL
database, and I've discovered that the way you reason about row and
table locks is very similar to the way you reason about locking among
threads. The big difference is the consequence of incorrect locking: in
PostgreSQL, using the serializable mode, incorrect locking generally
only leads to aborted transactions; while in Python and most programming
languages, incorrect locking instantly causes corruption and chaos.
That's what hurts developers. I want a concurrency model in Python that
acknowledges the need for locking while punishing incorrect locking with
an exception rather than corruption. *That* would be cool, IMHO.
More information about the Python-Dev