[Numpy-discussion] NumPy re-factoring project

[Dag Sverre Seljebotn]

Christopher Barker wrote:
> David Cournapeau wrote:
>>> In the core C numpy library there would be new  "numpy_array" struct
>>> with attributes
>>>
>>>  numpy_array->buffer
>
>> Anything non trivial will require memory allocation and object
>> ownership conventions.
>
> I totally agree -- I've been thinking for a while about a core array
> data structure that you could use from C/C++, and would interact well
> with numpy and Python -- it would be even better if it WAS numpy.
>
> I was thinking that at the root of it would be a "data_block" object
> (the buffer in the above), that would have a reference counting system.
> It would be its own system, but hopefully be able to link to Python's
> easily when used with Python.

I think taking PEP 3118, strip out the Python-specific things, and then
add memory management conventions, would be a good starting point.

Simply a simple header file/struct definition and specification, which
could in time become a de facto way of exporting multidimensional array
data between C libraries, between Fortran and C and so on (Kurt Smith's
fwrap could easily be adapted to support it). The C-NumPy would then be a
library on top of this spec (mainly ufuncs operating on such structs).

The memory management conventions needs some thought, as you say, because
of slices -- but a central memory allocator is not good enough because one
would often be accessing memory that's allocated with other purposes in
mind (and should not be deallocated, or deallocated in a special way). So
refcounts + deallocator callback seems reasonable.

(Not that I'm involved in this, just my 2 cents.)

Dag Sverre