[Python-ideas] Mutating while iterating

[Aaron Brady]

On Sun, Aug 3, 2014 at 12:46 PM, Aaron Brady <castironpi at gmail.com> wrote:
> On Sat, Jul 26, 2014 at 10:39 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
[snip]
>> Hi,
>>
>> This is clearly an issue of grave concern to you, but as Raymond
>> pointed out previously, you appear to have misunderstood the purpose
>> of those exceptions. They're there to prevent catastrophic failure of
>> the interpreter itself (i.e. segmentation faults), not to help find
>> bugs in user code. If users want to mutate containers while they're
>> iterating over them, they're generally free to do so. The only time
>> we'll actively disallow it is when such mutation will outright *break*
>> the iterator, rather than merely producing potentially surprising
>> results.
>>
>> I have closed the new issue and added a longer reply (with examples)
>> that will hopefully better explain why we have no intention of
>> changing this behaviour: http://bugs.python.org/issue22084#msg224100
>
>
> Python is replete with examples of prohibiting structures which are
> likely bugs but aren't segfaults.  There are also reciprocal-- though
> not necessarily inverse-- limitations of the unordered collections
> themselves ("set" and "dict").  The new behavior is transparent for
> the programmer; no possible programs can /rely/ on the existing
> behavior.  The new behavior introduces no new objects, possibly except
> "IterationError", no new syntax, and no new costs.
>
> I propose we leave this discussion thread open for the time being.  I
> also take the issue of /re/-assigning to keys during iteration to be
> settled as permitted.

Nick, It works by comparing the state of the container, to its state
when the iterator was opened.  We're ensuring it will always have a
unique state, up to comparison.  A state can be reused once no
iterators refer to it, hence needing the reference count.  A full
"object" is not needed for a memo, only the reference count, but the
"object" is easier and only twice the size, as "PyBaseObject Type" is
allocated anyway.

I'll point out that among the additional costs that there aren't,
garbage collection isn't any slower, as both "tp traverse" and "tp
clear" are empty in the "PyBaseObject Type" definition, on line
3511-3512 in "typeobject.c" at time of writing [1].

[1] http://svn.python.org/view/python/trunk/Objects/typeobject.c?revision=81744&view=markup#l3511