Creating an ndarray from an iterable over sequences
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32)  ValueError Traceback (most recent call last) <ipythoninput265bcc2e94dbca> in <module>() > 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Creating two 1dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memoryefficient but slow. List comprehension is vice versa.
If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences?
Leo
Hi,
On Tue, Jan 21, 2014 at 8:34 AM, Dr. Leo fhaxbox66@googlemail.com wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32)
ValueError Traceback (most recent call last) <ipythoninput265bcc2e94dbca> in <module>() > 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Perhaps you could just utilize structured arrays ( http://docs.scipy.org/doc/numpy/user/basics.rec.html), like: iterable= ((i, i**2) for i in range(10)) a= np.fromiter(iterable, [('a', int32), ('b', int32)], 10) a.view(int32).reshape(1, 2) Out[]: array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]])
My 2 cents, eat
Creating two 1dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memoryefficient but slow. List comprehension is vice versa.
If there is no solution, wouldn't it be possible to rewrite fromiter so as to accept sequences?
Leo
NumPyDiscussion mailing list NumPyDiscussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpydiscussion
On Tue, Jan 21, 2014 at 07:34:19AM +0100, Dr. Leo wrote:
Hi,
I would like to write something like:
In [25]: iterable=((i, i**2) for i in range(10))
In [26]: a=np.fromiter(iterable, int32)
ValueError Traceback (most recent call last) <ipythoninput265bcc2e94dbca> in <module>() > 1 a=np.fromiter(iterable, int32)
ValueError: setting an array element with a sequence.
Is there an efficient way to do this?
Creating two 1dimensional arrays first is costly as one has to iterate twice over the data. So the only way I see is creating an empty [10,2] array and filling it row by row. This is memoryefficient but slow. List comprehension is vice versa.
You could use itertools:
from itertools import chain g = ((i, i**2) for i in range(10)) import numpy numpy.fromiter(chain.from_iterable(g), numpy.int32).reshape(1, 2)
array([[ 0, 0], [ 1, 1], [ 2, 4], [ 3, 9], [ 4, 16], [ 5, 25], [ 6, 36], [ 7, 49], [ 8, 64], [ 9, 81]], dtype=int32)
Oscar
participants (3)

Dr. Leo

eat

Oscar Benjamin