[Python-Dev] can this overflow (list insertion)?

[Tim Peters]

[Trent Mick]
> from Objects/listobject.c:
>
> static int
> ins1(PyListObject *self, int where, PyObject *v)
> {
>     ...
>     self->ob_size++;         <-------------- can this overflow?
>     return 0;
> }

> ...
> Is it or was this relying on sizeof(ob_size) == sizeof(void*),
> hence a list being able to hold as many items as there is
> addressable memory?

I think it's more relying on the product of two other assumptions:  (a)
sizeof(int) >= 4, and (b) nobody is going to make a list with 2 billion
elements in Python.  But you're right, sooner or later that's going to bite
us.

> proposed patch:
>
> *** python/dist/src/Objects/listobject.c Fri Aug 11 16:25:08 2000
> --- Python/dist/src/Objects/listobject.c Fri Aug 11 16:25:36 2000
> ***************
> *** 149,155 ****
>         Py_INCREF(v);
>         items[where] = v;
>         self->ob_item = items;
> !       self->ob_size++;
>         return 0;
>   }
>
> --- 149,159 ----
>         Py_INCREF(v);
>         items[where] = v;
>         self->ob_item = items;
> !       if (self->ob_size++ == INT_MAX) {
> !               PyErr_SetString(PyExc_OverflowError,
> !                       "cannot add more objects to list");
> !               return -1;
> !       }
>         return 0;
>   }

+1 on catching it, -1 on this technique.  You noted later that this will
make trash out of ob_size if it triggers, but the list has already grown and
been shifted by this time too, so it's left in an insane state (to the user,
the last element will appear to vanish).

Suggest checking at the *start* of the routine instead:

       if (self->ob_size == INT_MAX) {
              PyErr_SetString(PyExc_OverflowError,
                      "cannot add more objects to list");
              return -1;
      }

Then the list isn't harmed.