[Numpy-discussion] ragged array implimentation

[Francesc Alted]

A Thursday 10 March 2011 20:29:00 Christopher Barker escrigué:
> I don't think so -- my approach is a lot simpler that I think carray
> is:
> 
> it acts much like a python list:
> 
> 1) it pre-allocates extra space when an array is created.

same for carray

> 2) when you append, it uses that extra space

ditto for carray

> 3) when the extra space is used up, it re-allocates the entire array,
> with some more extra room

again, carray works exactly the same: the extra room is just a new chunk

> And to keep it really simple, I'm using a numpy array internally, and
> np.ndarray.resize to grow it. It can only be grown on the first axis
> (C-order).

uh, very similar, except that due to the chunked nature of carray, it 
does not use `ndarray.resize`, but simply allocates a new chunk.

> > Perhaps it is a good time to join efforts and create a nice
> > 'growable' array package?
> 
> Sure, but I don't know that I have much to offer, except maybe some
> test and timing code.

Well, that sounds like it could be a start.  Your experience with your 
package would be a nice thing to have.  BTW, can you remember me where 
it is your code?  Maybe I can get some ideas by looking at it.

> By the way, it would be great to have a growable array that could be
> efficiently used in Cython code -- so you could accumulate a bunch of
> native C datatype data easily.

As you noticed, chunks and carrays are both native Cython extensions, so 
it should be pretty easy to call them from Cython directly, avoiding the 
Python overhead.

-- 
Francesc Alted