[Python-Dev] What do we do about bad slicing and possible crashes (issue 27867)

[Nick Coghlan]

On 31 August 2016 at 05:06, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 30.08.16 20:42, Nick Coghlan wrote:
>> Given Serhiy's clarification that this is primarily a thread safety
>> problem, I'm more supportive of the "PySlice_GetIndicesForObject"
>> approach (since that can call all the __index__ methods first, leaving
>> the final __len__ call as the only problematic case).
>
> This doesn't work with multidimensional slicing (like _testbuffer.ndarray or
> NumPy arrays).

Thanks, that makes sense.

>> However, given the observation that __len__ can also release the GIL,
>> I'm not clear on how 2A is supposed to work - a poorly timed thread
>> switch means there's always going to be a risk of len(obj) returning
>> outdated information if a container implemented in Python is being
>> mutated concurrently from different threads, so what can be done
>> differently in the calling functions that couldn't be done in a new
>> API that accepted the container reference?
>
> Current code doesn't use __len__. It uses something like
> PyUnicode_GET_LENGTH().

Oh, I see - it's the usual rule that C code can be made implicitly
atomic if it avoids calling hooks potentially written in Python, but
pure Python containers need explicit locks to allow concurrent access
from multiple threads.

> The solution was found easier than I afraid. See my patch to issue27867.

+1 from me. Would it make sense to make these public and new additions
to the stable ABI for 3.6+?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia