[Numpy-discussion] numpy all unexpected result (generator)

[Benjamin Root]

On Tue, Jan 31, 2012 at 10:05 AM, Olivier Delalleau <shish at keba.be> wrote:

> Le 31 janvier 2012 10:50, Robert Kern <robert.kern at gmail.com> a écrit :
>
>> On Tue, Jan 31, 2012 at 15:35, Benjamin Root <ben.root at ou.edu> wrote:
>> >
>> >
>> > On Tue, Jan 31, 2012 at 9:18 AM, Robert Kern <robert.kern at gmail.com>
>> wrote:
>> >>
>> >> On Tue, Jan 31, 2012 at 15:13, Benjamin Root <ben.root at ou.edu> wrote:
>> >>
>> >> > Is np.all() using np.array() or np.asanyarray()?  If the latter, I
>> would
>> >> > expect it to return a numpy array from a generator.
>> >>
>> >> Why would you expect that?
>> >>
>> >> [~/scratch]
>> >> |37> np.asanyarray(i>5 for i in range(10))
>> >> array(<generator object <genexpr> at 0xdc24a08>, dtype=object)
>> >>
>> >> --
>> >> Robert Kern
>> >
>> >
>> > What possible use-case could there be for a numpy array of generators?
>>
>> Not many. This isn't an intentional feature, just a logical
>> consequence of all of the other intentional features being applied
>> consistently.
>>
>> > Furthermore, from the documentation:
>> >
>> > numpy.asanyarray = asanyarray(a, dtype=None, order=None, maskna=None,
>> > ownmaskna=False)
>> >      Convert the input to an ndarray, but pass ndarray subclasses
>> through.
>> >
>> >      Parameters
>> >      ----------
>> >      a : array_like
>> >          Input data, in any form that can be converted to an array.
>> This
>> >          includes scalars, lists, lists of tuples, tuples, tuples of
>> tuples,
>> >          tuples of lists, and ndarrays.
>> >
>> > Emphasis mine.  A generator is an input that could be converted into an
>> > array.  (Setting aside the issue of non-terminating generators such as
>> those
>> > from cycle()).
>>
>> I'm sorry, but this is not true. In general, it's too hard to do all
>> of the magic autodetermination that asarray() and array() do when
>> faced with an indeterminate-length iterable. We tried. That's why we
>> have fromiter(). By restricting the domain to an iterable yielding
>> scalars and requiring that the user specify the desired dtype,
>> fromiter() can figure out the rest.
>>
>> Like it or not, "array_like" is practically defined by the behavior of
>> np.asarray(), not vice-versa.
>
>
> In that case I agree with whoever said ealier it would be best to detect
> this case and throw an exception, as it'll probably save some headaches.
>
> -=- Olivier
>
>
I'll agree with this statement.  This bug has popped up a few times in the
mpl bug tracker due to the pylab mode.  While I would prefer if it were
possible to evaluate the generator into an array, silently returning True
incorrectly for all() and any() is probably far worse.

That said, is it still impossible to make np.all() and np.any() special to
have similar behavior to the built-in all() and any()?  Maybe it could
catch the above exception and then return the result from python's
built-ins?

Cheers!
Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120131/9aa56e31/attachment.html>