Re: [Numpy-discussion] array comprehension
Are you sure fromiter doesn't make an intermediate list or equivalent? It has to collect all the values before it can know the shape or dtype of the array to put them in. On Nov 4, 2016 5:26 AM, "Francesc Alted" <faltet@gmail.com> wrote: 2016-11-04 13:06 GMT+01:00 Neal Becker <ndbecker2@gmail.com>:
Right. Using a generator and np.fromiter() will avoid the creation of the intermediate list. Something like: np.fromiter((i for i in range(x))) # use xrange for Python 2
-- Francesc Alted _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
On Fri, Nov 4, 2016 at 10:24 AM, Nathaniel Smith <njs@pobox.com> wrote:
fromiter dynamically resizes a NumPy array, like a Python list, except with a growth factor of 1.5 (rather than 1.25): https://github.com/numpy/numpy/blob/bb59409abf5237c155a1dc4c4d5b31e4acf32fbe...
On Nov 4, 2016 10:32 AM, "Stephan Hoyer" <shoyer@gmail.com> wrote:
On Fri, Nov 4, 2016 at 10:24 AM, Nathaniel Smith <njs@pobox.com> wrote:
Are you sure fromiter doesn't make an intermediate list or equivalent?
It has to collect all the values before it can know the shape or dtype of the array to put them in.
fromiter dynamically resizes a NumPy array, like a Python list, except
with a growth factor of 1.5 (rather than 1.25):
https://github.com/numpy/numpy/blob/bb59409abf5237c155a1dc4c4d5b31e4acf32fbe... Oh, right, and the dtype argument is mandatory, which is what makes this possible. -n
On Fri, Nov 4, 2016 at 10:36 AM, Nathaniel Smith <njs@pobox.com> wrote:
Oh, right, and the dtype argument is mandatory, which is what makes this possible.
Couldn't it determine the dtype from the first element, and then barf later if an incompatible one shows up? And then we could adapt this code to np.array() and get nice performance with no extra functions to think about calling... And off the top of my head, I can't think of why it couldn't be generalized to the nd case as well. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
On Fri, Nov 4, 2016 at 10:24 AM, Nathaniel Smith <njs@pobox.com> wrote:
fromiter dynamically resizes a NumPy array, like a Python list, except with a growth factor of 1.5 (rather than 1.25): https://github.com/numpy/numpy/blob/bb59409abf5237c155a1dc4c4d5b31e4acf32fbe...
On Nov 4, 2016 10:32 AM, "Stephan Hoyer" <shoyer@gmail.com> wrote:
On Fri, Nov 4, 2016 at 10:24 AM, Nathaniel Smith <njs@pobox.com> wrote:
Are you sure fromiter doesn't make an intermediate list or equivalent?
It has to collect all the values before it can know the shape or dtype of the array to put them in.
fromiter dynamically resizes a NumPy array, like a Python list, except
with a growth factor of 1.5 (rather than 1.25):
https://github.com/numpy/numpy/blob/bb59409abf5237c155a1dc4c4d5b31e4acf32fbe... Oh, right, and the dtype argument is mandatory, which is what makes this possible. -n
On Fri, Nov 4, 2016 at 10:36 AM, Nathaniel Smith <njs@pobox.com> wrote:
Oh, right, and the dtype argument is mandatory, which is what makes this possible.
Couldn't it determine the dtype from the first element, and then barf later if an incompatible one shows up? And then we could adapt this code to np.array() and get nice performance with no extra functions to think about calling... And off the top of my head, I can't think of why it couldn't be generalized to the nd case as well. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
participants (3)
-
Chris Barker -
Nathaniel Smith -
Stephan Hoyer