[Numpy-discussion] numpy, Py_ssize_t, cython and 64 bits python 2.4

[Charles R Harris]

On Sun, Nov 9, 2008 at 2:17 PM, Robert Kern <robert.kern at gmail.com> wrote:

> On Sun, Nov 9, 2008 at 14:01, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> > On Sun, Nov 9, 2008 at 11:44 AM, David Cournapeau <cournape at gmail.com>
> > wrote:
> >>
> >> On Mon, Nov 10, 2008 at 2:40 AM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >> >
> >> >
> >> > Let me see if I understand this correctly. For Python < 2.5 the list
> >> > indices
> >> > and such are ints, while for later versions they are Py_ssize_t, which
> >> > is
> >> > larger on 64 bit systems. Meanwhile, Py_intptr_t is large enough to
> hold
> >> > a
> >> > pointer.
> >>
> >> yes
> >>
> >> > So why are these two numbers being mixed?
> >>
> >> It is note that they are being mixed, but that cython does not support
> >> this configuration: it has a internal check which raise an exception
> >> in such a case. See around line 55:
> >>
> >> http://hg.cython.org/cython/file/764f1578df40/Cython/Includes/numpy.pxd
> >>
> >> As I understand, this means you can't use cython for such a
> >> configuration, but I just wanted to confirm whether there were known
> >> workarounds.
> >
> > Lessee,
> >
> > cdef extern from "Python.h":
> >      ctypedef int Py_intptr_t
> >
> > cdef extern from "numpy/arrayobject.h":
> >      ctypedef Py_intptr_t npy_intp
> > So they are screwing with the npy_intp type. They should hang.
>
> Chuck, you're being rude. This would be somewhat mitigated (at least
> from my perspective) if you knew what you are talking about. However,
> it appears that you don't.
>
> For integer types, the exact ctypedefs don't actually matter. The same
> code would probably be generated if you used "short", even. All these
> do is tell Cython to convert them to/from a Python int if they cross
> the Python boundary.
>

Crossing the boundary is one of the main reasons to use Cython, especially
for converting arguments in calling C functions. Numpy has the same problem
and, IIRC, gets around it by using PyObject_AsLong and casting the result
when needed. It isn't perfect, but there you go. To get cython to do this,
you can do:

ctypedef long npy_intp

And then

def hello(m) :
    cdef npy_intp m_ = <npy_intp>m
....

Does the right thing.

But mixing python types and numpy types is not a good idea, they exist
separately and apply to different software. This may be different for the
buffer interface, which is likely to cross the boundary, but that is an
argument for being very careful on how the buffer interface is dealt with in
cython.


> > Numpy is
> > numpy, Python is python, and never the two should meet. Note that none of
> > this crap is in the c_numpy.pxd included with numpy, BTW. I'd send the
> > cython folks a note and tell them to knock it off, the Py_* values are
> > irrelevant to numpy.
>
> And this is entirely irrelevant to the problem at hand.
>

Perhaps the immediate problem, but it is introducing an ugly dependency if
they aren't kept separate.


> David, this exception is explicitly raised in __getbuffer__() defined
> in numpy.pxd. It is a Cython limitation. Cython's buffer support
> cannot be used with numpy in Python < 2.5 currently. Ask Dag about it.
>

Yes, David referenced that line in numpy.pxd.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20081109/3139968d/attachment.html>