[Python-ideas] Fwd: Concurrent safety?

[Mike Meyer]

On Mon, Oct 31, 2011 at 11:55 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 11/1/2011 1:32 AM, Mike Meyer wrote:
>> As you point out, this is a hard problem. I know I haven't covered all
>> the issues. That's why the second thing I said was that I'm hoping to
>> get people smarter than me to look at things.
>
> This is one of the hard problems that keep getting swept under the rug while we do easier things. Well, we have overhauled unicode and packaging for 3.3, so maybe concurrency can get some attention.

Hey, it worked!

>
> I keep thinking that CPython's design of allowing C coded modules either outside or inside the stdlib should allow some progress.
>
> Would it be helpful, for instance, to have truly immutable restricted tuples and frozensets, whose __new__ methods only allowed true immutables (None, booleans, numbers, strings, other restricted tuples and frozensets) as members?

Possibly. However, so long as the mutations they make don't change the
externally visible behavior, then for the purposes of this discussion,
they already are immutable. Or is it possible that concurrent updates
of that not-externally-visible state could cause things to break?

> How about a metaclass, say 'immutable', that made the instances of a user class truly immutable? (I don't know how to do this, but lets assume it can be done -- perhaps with a new frozendict.) If such were possible, instances of instances of such a metaclass could be added to the list above.

Well, on the basis that we're all adults, I'm willing to accept that a
programmer saying "I want instances of this class to be immutable"
means they'll only subvert whatever mechanism is used to do this when
it's safe to do so (i.e. - "not externally visible"), so catching
casual attempts - assignments to attributes - to do so will do, then
we can do this by providing a __setattr__ method that always throws an
exception.

Actually, I think that's the key to implementing this efficiently.
__setattr__ on objects that aren't locked throws an exception (or
triggers locking inside an STM). Locking them changes __setattr__ to
something that works appropriately. Builtin types will need more
extensive tweaking along those lines. An immutable type doesn't need
the working variant of __setattr__.

> Could a metaclass automatically add fine-grained locks around around attribute mutations?

Wouldn't that be another variation on the __setattr__ method, that did:

    locking self.__dict__:
       self.__dict__[name] = value

I can see that that would be useful, but would expect most objects
would want to change more than one attribute in a consistent method,
so they'd have a method that locked self and made all those changes.

     <mike