[Cython] Python array support (#113)

[Stefan Behnel]

mark florisson, 06.05.2012 11:05:
> On 6 May 2012 09:56, mark florisson wrote:
>> On 5 May 2012 20:50, Stefan Behnel wrote:
>>>>   https://github.com/cython/cython/pull/113
>>>
>>> This looks ok to me now. There have been objections back when we discussed
>>> the initial patch for array.array support, so what do you think about
>>> merging this in?
>>
>> Great, I think it can be quite useful for some people. I think only
>> some documentation is missing.
>>
>> Also, very minor complaint, the way it allocates shape, strides and
>> the format string in __getbuffer__ is weird and complicated for no
>> good reason. I think it's better to declare two Py_ssize_t scalars or
>> one-sized arrays as class attributes and one char[2] array, and use
>> those (then you can also get rid of __releasebuffer__).
> 
> Or hm, that might be a problem with their variable nature? Then the
> shape of the buffer would suddenly change...

I'm fine with saying that any user who changes the size of an array while a
buffer view on it is being held (and used) is just plain out of warranty.
After all, a realloc() is allowed to move the memory buffer around and may
really do it in some cases, so the length is the least of our problems,
even if the array doesn't shrink but only grows.

I just noticed that the array module supports the buffer interface natively
in Python 3. That makes this whole patch somewhat less interesting, because
it's essentially just a work-around for a missing feature in Py2.

Py3 does the setup like this:

    view->buf = (void *)self->ob_item;
    view->obj = (PyObject*)self;
    Py_INCREF(self);
    if (view->buf == NULL)
        view->buf = (void *)emptybuf;
    view->len = (Py_SIZE(self)) * self->ob_descr->itemsize;
    view->readonly = 0;
    view->ndim = 1;
    view->itemsize = self->ob_descr->itemsize;
    view->suboffsets = NULL;
    view->shape = NULL;
    if ((flags & PyBUF_ND)==PyBUF_ND) {
        view->shape = &((Py_SIZE(self)));
    }
    view->strides = NULL;
    if ((flags & PyBUF_STRIDES)==PyBUF_STRIDES)
        view->strides = &(view->itemsize);
    view->format = NULL;
    view->internal = NULL;
    if ((flags & PyBUF_FORMAT) == PyBUF_FORMAT)
        view->format = self->ob_descr->formats;

It also counts the number of exports and prevents resizing while a buffer
is being exported. The current .pxd implementation cannot achieve that,
which is really unfortunate. ISTM that it should fall through to the native
buffer interface if the underlying array supports it.

Stefan