[Python-Dev] Pythonic concurrency
shane at hathawaymix.org
Sat Oct 8 00:12:13 CEST 2005
Antoine Pitrou wrote:
>>I'd be happy to explain how
>>ZODB solves those problems, if you're interested.
> Well, yes, I'm interested :)
> (I don't anything about Zope internals though, and I've never even used
Ok. Quoting your list:
> To apply the same thing to Python you would at least need :
> 1. a way to define a subset of the current bag of reachable objects
> which has to stay consistent w.r.t. transactions that are applied
> to it (of course, you would have several such subsets in any
> non-trivial application)
ZODB holds a tree of objects. When you add an attribute to an object
managed by ZODB, you're expanding the tree. Consistency comes from
- Each thread has its own lazy copy of the object tree.
- The application doesn't see changes to the object tree except at
- The ZODB store keeps old revisions, and the new MVCC feature lets
the application see the object system as it was at the beginning of the
- If you make a change to the object tree that conflicts with a
concurrent change, all changes to that copy of the object tree are aborted.
> 2. a way to start and end a transaction on a bag of objects (begin /
> commit / rollback)
ZODB includes a transaction module that does just that. In fact, the
module is so useful that I think it belongs in the standard library.
> 3. a precise definition of the semantics of "consistency" here : for
> example, only one thread could modify a bag of objects at any given
> time, and other threads would continue to see the frozen,
> stable version of that bag until the next version is committed by the
> writing thread
As mentioned above, the key is that ZODB maintains a copy of the objects
per thread. A fair amount of RAM is lost that way, but the benefit in
simplicity is tremendous.
You also talked about the risk that applications would accidentally pull
a lot of objects into the tree just by setting an attribute. That can
and does happen, but the most common case is already solved by the
pickle machinery: if you pickle something global like a class, the
pickle stores the name and location of the class instead of the class
More information about the Python-Dev