[Numpy-discussion] Numpy on Python3
Dag Sverre Seljebotn
dagss at student.matnat.uio.no
Mon Nov 23 13:49:38 EST 2009
Pauli Virtanen wrote:
> Mon, 23 Nov 2009 08:58:47 +0100, Sturla Molden wrote:
>> Pauli Virtanen skrev:
>>> XXX: 3K: numpy.random is disabled for now, uses PyString_* XXX: 3K:
>>> numpy.ma is disabled for now -- some issues
>> I thought numpy.random uses Cython? Is it just a matter of recompiling
>> the pyx-file?
> The Cython file uses the C-api directly, so we'll need a .h file with the
> necessary compile-time conditionals.
>>> I remember Dag was working on this a bit: how far did it go?
>> Cython's include file numpy.pxd has an ndarray class that extend
>> PyArrayObject with PEP 3118 buffer compatibility.
> Great! I believe I will just steal whatever I can and rewrite it in C --
> for now, it seems possible to keep Numpy's core in plain C.
I did sit down with David to learn enough to do this and had a brief
start on doing it properly for NumPy on SciPy 2009 (with seperate
testcases and the buffer format string stored directly in the NumPy
dtype structs on creation). I meant to come back to it in November but
due to becoming sick etc. etc. that's no longer possible. If nothing
happens by the mid/end of January I still hope to be able to do this
then. Feel free to ask any questions about the buffer PEP if you do go
forward with this as I've used it a lot in Cython (and wrote the
"implementation on behalf of NumPy" there).
The Cython numpy.pxd does:
- Temporarily allocate buffers for the format string, this is
inefficient as NumPy can store them directly in the dtype when the dtype
- Not support non-native endian (and some other relevant packing
formats I believe)
- Not support the string types
- Not support dtypes with nested sub-arrays within records
What is done: David added some code (at least in some branch) that
ensures that sizeof(npy_intp) == sizeof(Py_ssize_t) on Python 2.6+.
(I.e. if that assumption is violated NumPy won't compile, so we're free
to assume it until the issue of using npy_intp for indices is fixed on a
more fundamental level in NumPy). This means that the shapes/strides in
Py_buffer can be directed directly to the dimensions/strides in the
NumPy array struct (whereas Cython's numpy.pxd has to make a copy on
some platforms for Python 2.4).
More information about the NumPy-Discussion