[Cython] question on Get/SetItemInt*() utility functions

[Stefan Behnel]

Hi,

the Get/SetItemInt helper macros usually look something like this:

"""
#define __Pyx_GetItemInt(o, i, size, to_py_func, is_list, wraparound,
boundscheck) \
    (((size) <= sizeof(Py_ssize_t)) ? \
    __Pyx_GetItemInt_Fast(o, i, is_list, wraparound, boundscheck) : \
    __Pyx_GetItemInt_Generic(o, to_py_func(i)))
"""

e.g. here:

https://github.com/cython/cython/blob/59c3408c054bbbb7334ccca7d76bed93f26057e9/Cython/Utility/ObjectHandling.c#L248

https://github.com/cython/cython/blob/59c3408c054bbbb7334ccca7d76bed93f26057e9/Cython/Utility/StringTools.c#L318

The "generic" part will almost never be called, except when the index
*type* is really huge. However, if that happens, there's a rather large
impact due to object creation, even if the value in question is tiny.

Is there a way to safely extend this compile time size check with checks
that compare the actual runtime value with PY_SSIZE_T_MIN and PY_SSIZE_T_MAX ?

Within those bounds, a cast would do the trick just fine, and we could
still use the fast path. If it fails that bounds check, however, and the
type we are processing is a builtin container, we'd already know that an
exception is going to be raised and could simply drop the generic
implementation, replacing it with an appropriate IndexError directly.
Obviously, non-builtins would still need to fall back to the generic
implementation, but there is a lot of special casing code for builtins that
(IMHO) could be removed and replaced by a simple exception.

I'm aware that the C comparison rules for unsigned vs. signed might make
this tricky, but I also know that some on this list know their C better
than I do. :)

BTW, I also wonder if the above sizeof() comparison is safe enough - what
if "i" has an unsigned type of the same size as Py_ssize_t and its value is
larger than PY_SSIZE_T_MAX ? Wouldn't it wrap around in that case?

Stefan