Creating an ndarray from an iterable over sequences
![](https://secure.gravatar.com/avatar/1e6b242e020d0be0837f0585273bfd5a.jpg?s=120&d=mm&r=g)
Hi, I would like to write something like: In [25]: iterable=((i, i**2) for i in range(10)) In [26]: a=np.fromiter(iterable, int32) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32) ValueError: setting an array element with a sequence. Is there an efficient way to do this? Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa. If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences? Leo
![](https://secure.gravatar.com/avatar/32d0b01c2c9ff4abdad2e0f3f0041400.jpg?s=120&d=mm&r=g)
Hi, On Tue, Jan 21, 2014 at 8:34 AM, Dr. Leo <fhaxbox66@googlemail.com> wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Perhaps you could just utilize structured arrays ( http://docs.scipy.org/doc/numpy/user/basics.rec.html), like: iterable= ((i, i**2) for i in range(10)) a= np.fromiter(iterable, [('a', int32), ('b', int32)], 10) a.view(int32).reshape(-1, 2) Out[]: array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]]) My 2 cents, -eat
Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa.
If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences?
Leo
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
![](https://secure.gravatar.com/avatar/664d320baa05c827ff08ed361fe77769.jpg?s=120&d=mm&r=g)
On Tue, Jan 21, 2014 at 07:34:19AM +0100, Dr. Leo wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32) --------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-26-5bcc2e94dbca> in <module>() ----> 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Creating two 1-dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memory-efficient but slow. List comprehension is vice versa.
You could use itertools:
from itertools import chain g = ((i, i**2) for i in range(10)) import numpy numpy.fromiter(chain.from_iterable(g), numpy.int32).reshape(-1, 2) array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]], dtype=int32)
Oscar
participants (3)
-
Dr. Leo
-
eat
-
Oscar Benjamin