[Python-Dev] Issue 19332: Guard against changing dict during iteration

Nick Coghlan ncoghlan at gmail.com
Wed Nov 6 06:41:38 CET 2013


On 6 Nov 2013 15:02, "Ethan Furman" <ethan at stoneleaf.us> wrote:
>
> http://bugs.python.org/issue19332
>
> Summary:
>
> --> d = {1: 'one'}
> --> for k in d:
> ...   d[k+1] = d[k] * k
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> RuntimeError: dictionary changed size during iteration
>
> --> for k in d:
> ...   del d[k]
> ...
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> RuntimeError: dictionary changed size during iteration
>
>
> While the above two cases are covered, the following is not:
>
> --> d = dict.fromkeys('abcd')
> --> for i in d:
> ...     print(i)
> ...     d[i + 'x'] = None
> ...     del d[i]
> ...
> d
> a
> dx
> dxx
> ax
> c
> b
>
> Which can yield pretty bizarre results and does not raise an exception.
>
> There is a patch to fix this.
>
> Raymond's first comment was:
>
> "The decision to not monitor adding or removing keys was intentional.  It
is just not worth the cost in either time or space."
>
> Considering that the first two scenarios do raise exceptions that
statement does not seem correct, at least not with current code.
>
> Later, Raymend rejected the patch with the comment:
>
> "I disagree with adding such unimportant code to the critical path."
>
> I understand we need to keep the critical path as fast as possible, and
that it is a balancing act between completely correct code and warnings in
the docs.
>
> What I'm hoping for is either someone to shed some light on what is the
"unimportant" part,

The behaviour of mutating builtin containers while iterating over them is
formally undefined beyond "it won't segfault" (one of the few such
undefined behaviours in Python). The associated exceptions are thus
strictly "best effort given other constraints".

Since most loops are unlikely to add and remove exactly balanced numbers of
keys, and dict read and write performance impacts essentially all Python
code except that which only operates on function locals, only checking for
size changes between iteration steps is a nice way to get 99% of the
benefits without paying any of the costs.

>or perhaps some performance measurements that would show there is no
performance based reason to not have the patch added.  (I can try to do the
performance part myself if necessary, I'm just not sure what all the steps
are yet.)

If the benchmark suite indicates there's no measurable speed penalty then
such a patch may be worth reconsidering. I'd be astonished if that was
actually the case, though - the lowest impact approach I can think of is to
check for live iterators when setting a dict entry, and that still has
non-trivial size and speed implications.

Cheers,
Nick.

>
> --
> ~Ethan~
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20131106/2ebde827/attachment.html>


More information about the Python-Dev mailing list