Creating an ndarray from an iterable over sequences
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa.
If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences?
Leo
Hi,
On Tue, Jan 21, 2014 at 8:34 AM, Dr. Leo fhaxbox66@googlemail.com wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32)
ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Perhaps you could just utilize structured arrays ( http://docs.scipy.org/doc/numpy/user/basics.rec.html), like: iterable= ((i, i**2) for i in range(10)) a= np.fromiter(iterable, [('a', int32), ('b', int32)], 10) a.view(int32).reshape(-1, 2) Out[]: array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]])
My 2 cents, -eat
Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa.
If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences?
Leo
NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Jan 21, 2014 at 07:34:19AM +0100, Dr. Leo wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32)
ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa.
You could use itertools:
from itertools import chain g = ((i, i**2) for i in range(10)) import numpy numpy.fromiter(chain.from_iterable(g), numpy.int32).reshape(-1, 2)
array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]], dtype=int32)
Oscar
participants (3)
-
Dr. Leo -
eat -
Oscar Benjamin