[Python-Dev] Issue 14417: consequences of new dict runtime error

[Etienne Robillard]

"Multiple threads can agree by convention not to mutate a shared dict, 
there's no great need for enforcement. Multiple processes can't share 
dicts."

its not sure I get completely the meaning of "mutate"...  And if 
possible, I would like also the rational for the 2nd phrase while we're 
at it as it seem a little unclear too. Sorry also
if this is OT... :)

Regards,
Etienne

http://www.python.org/dev/peps/pep-0416/

On 03/30/2012 10:47 AM, Guido van Rossum wrote:
> Etienne, I have not understood either of your messages in this thread.
> They just did not make sense to me. Do you actually understand the issue
> at hand?
>
> --Guido
>
> On Friday, March 30, 2012, Etienne Robillard wrote:
>
>     On 03/29/2012 06:07 PM, R. David Murray wrote:
>
>         On Thu, 29 Mar 2012 23:00:20 +0200, Stefan
>         Behnel<stefan_ml at behnel.de>  wrote:
>
>             R. David Murray, 29.03.2012 22:31:
>
>                 On Thu, 29 Mar 2012 13:09:17 -0700, Guido van Rossum wrote:
>
>                     On Thu, Mar 29, 2012 at 12:58 PM, R. David Murray wrote:
>
>                         Some of us have expressed uneasiness about the
>                         consequences of dict
>                         raising an error on lookup if the dict has been
>                         modified, the fix Victor
>                         made to solve one of the crashers.
>
>                         I don't know if I speak for the others, but
>                         (assuming that I understand
>                         the change correctly) my concern is that there
>                         is probably a significant
>                         amount of threading code out there that assumes
>                         that dict *lookup* is
>                         a thread-safe operation.  Much of that code
>                         will, if moved to Python
>                         3.3, now be subject to random runtime errors for
>                         which it will not
>                         be prepared.  Further, code which appears safe
>                         can suddenly become
>                         unsafe if a refactoring of the code causes an
>                         object to be stored in
>                         the dictionary that has a Python equality method.
>
>
>                     My original assessment was that this only affects
>                     dicts whose keys
>                     have a user-implemented __hash__ or __eq__
>                     implementation, and that
>                     the number of apps that use this *and* assume the
>                     threadsafe property
>                     would be pretty small. This is just intuition, I
>                     don't have hard
>                     facts. But I do want to stress that not all dict
>                     lookups automatically
>                     become thread-unsafe, only those that need to run
>                     user code as part of
>                     the key lookup.
>
>
>                 You are probably correct, but the thing is that one
>                 still has to do the
>                 code audit to be sure...and then make sure that no one
>                 later introduces
>                 such an object type as a dict key.
>
>
>             The thing is: the assumption that arbitrary dict lookups are
>             GIL-atomic has
>             *always* been false. Only those that do not involve Python
>             code execution
>             for the hash key calculation or the object comparison are.
>             That includes
>             the built-in strings and numbers (and tuples of them), which
>             are by far the
>             most common dict keys. Looking up arbitrary user provided
>             objects is
>             definitely not guaranteed to be atomic.
>
>
>         Well, I'm afraid I was using the term 'thread safety' rather too
>         loosely
>         there.  What I mean is that if you do a dict lookup, the lookup
>         either
>         returns a value or a KeyError, and that if you get back an
>         object that
>         object has internally consistent state.  The problem this fix
>         introduces
>         is that the lookup may fail with a RuntimeError rather than a
>         KeyError,
>         which it has never done before.
>
>         I think that is what Guido means by code that uses objects with
>         python
>         eq/hash *and* assumes threadsafe lookup.  If mutation of the objects
>         or dict during the lookup is a concern, then the code would use
>         locks
>         and wouldn't have the problem.  But there are certainly situations
>         where it doesn't matter if the dictionary mutates during the lookup,
>         as long as you get either an object or a KeyError, and thus no
>         locks are
>         (currently) needed.
>
>         Maybe I'm being paranoid about breakage here, but as with most
>         backward
>         compatibility concerns, there are probably more bits of code
>         that will
>         be affected than our intuition indicates.
>
>         --David
>         ______________________________ _________________
>
>
>     what this suppose to mean exactly? To "mutate" is a bit odd concept
>     for a programming language I suppose. Also I suppose I must be
>     missing something which makes you feel like this is an OT post when
>     the problem seem most likely to be exclusively in python 3.3,
>     another reason I guess to not upgrade yet all that massively using
>     2to3. :-)
>
>     cheers,
>     Etienne
>     ______________________________ _________________
>     Python-Dev mailing list
>     Python-Dev at python.org
>     http://mail.python.org/ mailman/listinfo/python-dev
>     <http://mail.python.org/mailman/listinfo/python-dev>
>     Unsubscribe: http://mail.python.org/ mailman/options/python-dev/
>     guido%40python.org
>     <http://mail.python.org/mailman/options/python-dev/guido%40python.org>
>
>
>
> --
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)