[Numpy-discussion] numpy, Py_ssize_t, cython and 64 bits python 2.4

[Dag Sverre Seljebotn]

Charles R Harris wrote:
>
>
> On Sun, Nov 9, 2008 at 4:29 PM, Dag Sverre Seljebotn 
> <dagss at student.matnat.uio.no <mailto:dagss at student.matnat.uio.no>> wrote:
>
>
>     What the Cython numpy.pxd file does is implementing PEP 3118 [1],
>     which
>     is supported by Cython in all Python versions (ie backported, not
>     following any standard). And, in Py_buffer, the strides and shapes are
>     Py_ssize_t* (which is also backported as David mentions). So, in order
>     to cast the shape and stride arrays and return them in the Py_buffer
>     struct, they need to have the datatype defined by the backported PEP
>     3118, i.e. the backported Py_ssize_t, i.e. int.
>
>
> So the backported version is pretty much a cython standard?
Yes. The whole thing is described in 
http://wiki.cython.org/enhancements/buffer , under the section "Buffer 
acquisition and Python versions". Basically the PEP is followed for 
Python 2.6+, then some tricks are done to make a similar interface work 
for older Python versions.

Note especially that if you NumPy people set Py_TPFLAGS_HAVE_NEWBUFFER 
and provide a bf_getbuffer implementation following the exact 
(non-backported) , the __getbuffer__ in numpy.pxd will not come into 
play on Python 2.6+.

>
>     OTOH, one could also opt for changing how PEP 3118 is backported
>     and say
>     that "for Python 2.4 we say that Py_buffer has Py_intptr_t* fields
>     instead". This would be more work to get exactly right, and would be
>     more contrived as well, but is doable if one really wants to get
>     rid of
>     the extra mallocs.
>
>
> This would be the direct way. The check could then be if 
> sizeof(npy_intp) != sizeof(Py_intptr_t). That is more reasonable as 
> they are supposed to serve the same purpose. If numpy is the only user 
> of this interface that is the route I would go. Is there an official 
> description of how PEP 3118 is to be backported? I don't know who else 
> uses it at the moment.
NumPy is currently the only user that I know of, and is likely to remain 
so I think, so making this change is ok. The backport is very 
Cython-specific because of the Cython-specific calling convention 
(essentially, objects lack the needed slot in earlier Python versions, 
and we do not want to pay the price of a dict lookup) and is about as 
official as things get with Cython (i.e. a wikipage).

Dag Sverre