[Python-ideas] Optimistic Concurrency

[Leif Walsh]

On Thu, Oct 16, 2008 at 9:03 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Suppose shared data consists of mutable collections.
> Let the program units that share the date follow this protocol (discipline)
> for editing members of a collection.
>
> old_id = id(member)
> edit_copy = collection.member # or collection[index/key] or whatever
> edit edit_copy
> done = False
> while not done:
>  atomically:
>    if id(edit_copy) == old_id:
>      collection.member = copy # by reference copy, not data copy
>      done = True
>  if done: break
>  recover(edit_copy)
>
> 'Atomically' could be by a very short term lock or perhaps a builtin C API
> function if such can run 'atomically' (I just don't know).

It feels like, if a new syntax element 'atomically' is added, it would
need to know what variable or collection is meant to be atomic.  This
poses a problem for objects with other references elsewhere, but I
think (and I don't know the internal structures of python very well,
so this may be way off the mark), the lock could be applied at the
data level, not the pointer level, to alleviate this issue.

> I believe this will work if all editors edit at the same level of
> granularity.  Certainly, mixing levels can easily lead to data loss.

Again, I think we can fix this by applying the lock to the data.

For example (since we talked about this in a class I had today), think
about the vfs in linux:

Say we have one file with many hardlinks.  If we apply a lock to the
dentry for the file, and then the user tries to modify our file from a
different dentry, the lock won't function.  If we apply the lock to
the inode, however, it locks it against changes from all hardlinks.
Hopefully python has a similar opportunity.

> Or the program might ask a human user, which was the use-case for this idea.

I don't think I understood the use case.  Can you explain it?

-- 
Cheers,
Leif