[Numpy-discussion] fromiter

[Tim Hochberg]

Travis Oliphant wrote:

>Tim Hochberg wrote:
>  
>
>>Some time ago some people, myself including, were making some noise 
>>about having 'array' iterate over iterable object producing ndarrays in 
>>a manner analogous to they way sequences are treated. I finally got 
>>around to looking at it seriously and once I came to the following three 
>>conclusions:
>>
>>   1. All I really care about is the 1D case where dtype is specified.
>>      This case should be relatively easy to implement so that it's
>>      fast. Most other cases are not likely to be particularly faster
>>      than converting the iterators to lists at the Python level and
>>      then passing those lists to array.
>>   2. 'array' already has plenty of special cases. I'm reluctant to add
>>      more.
>>   3. Adding this to 'array' would be non-trivial. The more cases we
>>      tried to make fast, the more likely that some of the paths would
>>      be buggy. Regardless of how we did it though, some cases would be
>>      much slower than other, which would probably be suprising.
>>  
>>    
>>
>
>Good job.   I just added a called fromiter for this very purpose.  Right 
>now, it's just a stub that calls list(obj) first and then array.  Your 
>code would be a perfect fit for it.  I think count could be optional, 
>though, to handle cases where the count can be determined from the object.
>  
>
I'll look at that when I get back. There are two ways to approach this: 
one is to only allow to count to be optional in those cases that the 
original object supports either __len__ or __length_hint__. The 
advantage their is that it's easy and there's no chance of locking up 
the interpreter by passing an unbounded generator. The other way is to 
figure out the length based on the generator itself. The "natural" way 
to do this is to steal stuff from array.array. However, that doesn't 
export a C-level interface that I can tell (everything is declared 
static), so you'd be going through the interpreter, which would 
potentially be slow. I guess another approach would be to hijack 
PyArray_Resize and steal the resizing pattern from array.array. I'm not 
sure how well that would work though.

I'll look into it...

-tim


>We'll look forward to your check-in.
>
>-Travis
>
>
>
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>  
>