[Python-Dev] What do we do about bad slicing and possible crashes (issue 27867)

[Serhiy Storchaka]

On 30.08.16 15:31, Dima Tisnek wrote:
> On 30 August 2016 at 14:13, Serhiy Storchaka <storchaka at gmail.com> wrote:
>>> 1. Detect length change and raise.
>>
>>
>> It would be simpler solution. But I afraid that this can break third-party
>> code that "just works" now. For example slicing a list "just works" if step
>> is 1. It can return not what the author expected if a list grows, but it
>> never crashes, and existing code can depends on current behavior. This
>> solution is not applicable in maintained versions.
>
> Serhiy,
>
> If dictionary is iterated in thread1 while thread2 changes the
> dictionary, thread1 currently raises RuntimeError.
>
> Would cloning current dict behaviour to slice with overridden
> __index__ make sense?

No, these are different things. The problem with dict iterating is 
unavoidable, but slicing can be defined consistently (as described by 
Terry in option 2). Changing a dict can change the order and invalidates 
iterators (except trivial cases of just created or finished iterators). 
But slicing can be atomic (and it is atomic de facto in many cases), we 
just should call all __index__-es before looking on a sequence.

> I'd argue 3rd party code depends on slicing not to raise an exception,
> is same as 3rd party code depending on dict iteration not to raise and
> exception; If same container may be concurrently used in another
> thread, then 3rd party code is actually buggy. It's OK to break such
> code.

We shouldn't break third-party code in maintained releases. De facto 
slicing is atomic now in many cases, and it is nowhere documented that 
it is not atomic. The problem only with using broken by design 
PySlice_GetIndicesEx() in CPython. If CPython would implemented without 
PySlice_GetIndicesEx() (with more cumbersome code), it is likely this 
issue wouldn't be raised.