I'll try to get the code up on gitHub.

It would be nice to combine efforts.

-CHB

 

In my case I need to ensure a contiguous storage to allow easy upload onto the GPU.
But my implementation is quite slow, especially when you add one item at a time:

>>> python benchmark.py
Python list, append 100000 items: 0.01161
Array list, append 100000 items: 0.46854
Array list, append 100000 items at once: 0.05801
Python list, prepend 100000 items: 1.96168
Array list, prepend 100000 items: 12.83371
Array list, append 100000 items at once: 0.06002



I realize I did not answer all Chris' questions:

>>> L = ArrayList( [[0], [1,2], [3,4,5], [6,7,8,9]] )
>>> for item in L: print(item)
[0]
[1 2]
[3 4 5]
[6 7 8 9]

>>> print (type(L.data))
<class 'numpy.ndarray'>
>>> print(L.data.dtype)
int64
>>> print(L.data.shape)
(10,)


I did not implement operations yet, but it would be a matter for transferring call to the underlying numpy data array.
>>> L._data *= 2
>>> print(L)
[[0], [4 8], [12 16 20], [24 28 32 36]]

> On 23 Dec 2015, at 09:34, Stephan Hoyer <shoyer@gmail.com> wrote:
>
> We have a type similar to this (a typed list) internally in pandas, although it is restricted to a single dimension and far from feature complete -- it only has .append and a .to_array() method for converting to a 1d numpy array. Our version is written in Cython, and we use it for performance reasons when we would otherwise need to create a list of unknown length:
> https://github.com/pydata/pandas/blob/v0.17.1/pandas/hashtable.pyx#L99
>
> In my experience, it's several times faster than using a builtin list from Cython, which makes sense given that it needs to copy about 1/3 the data (no type or reference count for individual elements). Obviously, it uses 1/3 the space to store the data, too. We currently don't expose this object externally, but it could be an interesting project to adapt this code into a standalone project that could be more broadly useful.
>
> Cheers,
> Stephan
>
>
>
> On Tue, Dec 22, 2015 at 8:20 PM, Chris Barker <chris.barker@noaa.gov> wrote:
>
> sorry for being so lazy as to not go look at the project pages, but....
>
> This sounds like it could be really useful, and maybe supercise a coupl eof half-baked projects of mine. But -- what does "dynamic" mean?
>
> - can you append to these arrays?
> - can it support "ragged arrrays" -- it looks like it does.
>
> >>> L = ArrayList( [[0], [1,2], [3,4,5], [6,7,8,9]] )
> >>> print(L)
> [[0], [1 2], [3 4 5], [6 7 8 9]]
>
> so this looks like a ragged array -- but what do you get when you do:
>
> for row in L:
>     print row
>
>
> >>> print(L.data)
> [0 1 2 3 4 5 6 7 8
>
> is .data a regular old 1-d numpy array?
>
> >>> L = ArrayList( np.arange(10), [3,3,4])
> >>> print(L)
> [[0 1 2], [3 4 5], [6 7 8 9]]
> >>> print(L.data)
> [0 1 2 3 4 5 6 7 8 9]
>
>
> does an ArrayList act like a numpy array in other ways:
>
> L * 5
>
> L* some_array
>
> in which case, how does it do broadcasting???
>
> Thanks,
>
> -CHB
>
> >>> L = ArrayList(["Hello", "world", "!"])
> >>> print(L[0])
> 'Hello'
> >>> L[1] = "brave new world"
> >>> print(L)
> ['Hello', 'brave new world', '!']
>
>
>
> Nicolas
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker@noaa.gov
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov