[New-bugs-announce] [issue36452] Detect dict iteration "overflow" when changing keys

[Thomas Perl]

New submission from Thomas Perl <th.perl at gmail.com>:

Using: Python 3.8 (git commit ID: d5a5a33f12b60129d57f9b423b77d2fcba506834), the following code snippet:

=====
a = {0: 0}

for i in a:
    del a[i]
    a[i+1] = 0
    print(i)
=====

Prints the following output:

=====
0
1
2
3
4
=====

The reason for this seems to be the way the internal key list is managed and the "next" value in this list is retrieved. The amount of items seems to be related to USABLE_FRACTION(PyDict_MINSIZE).

Since cases where the dictionary size changes are detected with a RuntimeError, I would expect the invariant to be "the number of iterations is the len() of the dict at the time the iterator is created to be enforced. Whether to raise a StopIteration instead or raising a RuntimeError is up for debate.

Attached is a patch that tries to detect this corner case and raise a RuntimeError instead (plus a unit test).

Note also that without the patch, the __length_hint__() of the iterator actually underflows:

=====
a = {0: 0}
it = iter(a)
print('Length hint:', it.__length_hint__())
next(it)
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 0
next(it)
print('Length hint:', it.__length_hint__())
=====

----------
files: 0001-dictiterobject-Track-maximum-iteration-count-via-di-.patch
keywords: patch
messages: 338997
nosy: thomas.perl
priority: normal
severity: normal
status: open
title: Detect dict iteration "overflow" when changing keys
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file48234/0001-dictiterobject-Track-maximum-iteration-count-via-di-.patch

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36452>
_______________________________________