[Numpy-discussion] bad generator behaviour with sum

[Tom Denniston]

I was thinking about this in the context of Giudo's comments at scipy
2006 that much of the language is moving away from lists toward
iterators.  He gave the keys of a dict as an example.

Numpy treats iterators, generators, etc as 0x0 PyObjects rather than
lazy generators of n dimensional data.  I guess my question for Travis
(any others much more expert than I in numpy) is is this intentional
or is it something that was never implemented because of the obvious
subtlties of defiing the correct semantics to make this work.

Personally i find it no big deal to use array(list(iter)) in the 1d
case and the list function combined with a list comprehension for the
2d case.  I usually know how many dimensions i expect so i find this
easy and i know about this peculiar behavior.  I find, however, that
this behavior is very suprising and confusing to the new user and i
don't usually have a good justification for it to answer them.

The ideal semantics, in my mind, would be if an iterator of iterators
of iterators, etc was no different in numpy than a list of lists of
lists, etc.  But I have no doubt that there are subtleties i am not
considering. Has anyone more familiar than I with the bowels of numpy
thought about this problem and see reasons why this is a bad idea or
just prohibitively difficult to implement?

On 8/27/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> Hi,
>
> The problem seems to arise in the array constructor, which treats the
> generator as a python object and creates an array containing that object.
> So, do we want the possibility of an array of generators or should we
> interpret it as a sort of list? I vote for that latter.
>
> Chuck
>
>
> On 8/27/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
> >
> > Hi Christopher,
> >
> >
> >
> > On 8/27/06, Charles R Harris < charlesr.harris at gmail.com> wrote:
> > >
> > > Hi,
> > >
> > >
> > >
> > > On 8/27/06, listservs at mac.com <listservs at mac.com> wrote:
> > > > -----BEGIN PGP SIGNED MESSAGE-----
> > > > Hash: SHA1
> > > >
> > > > It seems like numpy.sum breaks generator expressions:
> > > >
> > > > In [1]: sum(i*i for i in range(10))
> > > > Out[1]: 285
> > > >
> > > > In [2]: from numpy import sum
> > > >
> > > > In [3]: sum(i*i for i in range(10))
> > > > Out[3]: <generator object at 0x10eca58>
> > > >
> > > > Is this intentional? If so, how do I get the behaviour that I am
> after?
> > > >
> > >
> > >
> > >
> > >
> > >
> > > In [3]: sum([i*i for i in range(10)])
> > >
> > > Out[3]: 285
> > >
> > > Chuck
> >
> >
> >
> > The numarray.sum also fails to accept a generator as an argument. Because
> python does and the imported sum overwrites it, we should probably check the
> argument type and make it do the right thing.
> >
> > Chuck
> >
> >
> >
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
>
>