[Python-ideas] Optimistic Concurrency

[Adam Olsen]

On Mon, Oct 20, 2008 at 8:31 AM,  <skip at pobox.com> wrote:
>    Leonardo> Another idea that maybe have a chance of working is
>    Leonardo> implementing Software Transactional Memory on the interpreter
>    Leonardo> level and use it to implement threading...
>
> That's the term I was looking for, but couldn't remember it.  I saw an
> article on STM a couple months ago and thought that might work for Python.
> Aren't "optimistic concurrency" and "software transactional memory" very
> similar concepts?

They're essentially the same.  Optimistic concurrency is one of the
tools used by STS.

To counter Terry's example, I believe it should look like this:

transaction:
    edit collection.member # or collection[index/key] or whatever

IOW, the collection container should automatically create a view when
you touch it, adding it to the current transaction.  When the
transaction block completes, the language attempts to atomically
commit all the views, restarting if there's a conflict.

This has substantial problems if you mix a very long transaction with
many short ones.  Various ways of prioritizing restarted transactions
are possible, at the risk of locking everybody out if the long
transaction takes too long (or never completes).

Also note that this doesn't require optimistic concurrency.  It's just
as valid to raise an exception when editing if another commit has
invalidated your transaction.  In fact, since views are added lazily,
this is more-or-less required.

Stepping back a bit, there's two distinct problems to solve:
1) Making threading easier
2) Removing the GIL

#1 can be done by things like monitors and transactions (the two are
complementary).  #2 at its core is about refcounting, and transactions
are irrelevant there — you need to switch to a tracing-based GC
implementation


-- 
Adam Olsen, aka Rhamphoryncus