Mailman 3 Faster array version of ndindex - NumPy-Discussion

newer
Problems installing numpy on Mac...

Faster array version of ndindex

older
user-defined types mixed arithmetic

Jonathan Taylor

13 Dec 2007 13 Dec '07

6:33 p.m.

I was needing an array representation of ndindex since ndindex only gives an iterator but array(list(ndindex)) takes too long. There is prob some obvious way to do this I am missing but if not feel free to include this code which is much faster.

In [252]: time a=np.array(list(np.ndindex(10,10,10,10,10,10))) CPU times: user 11.61 s, sys: 0.09 s, total: 11.70 s Wall time: 11.82

In [253]: time a=ndtuples(10,10,10,10,10,10) CPU times: user 0.32 s, sys: 0.21 s, total: 0.53 s Wall time: 0.60

def ndtuples(*dims): """Fast implementation of array(list(ndindex(*dims)))."""

# Need a list because we will go through it in reverse popping # off the size of the last dimension. dims = list(dims)

# N will keep track of the current length of the indices. N = dims.pop()

# At the beginning the current list of indices just ranges over the # last dimension. cur = np.arange(N) cur = cur[:,np.newaxis]

while dims != []:

d = dims.pop()

# This repeats the current set of indices d times. # e.g. [0,1,2] -> [0,1,2,0,1,2,...,0,1,2] cur = np.kron(np.ones((d,1)),cur)

# This ranges over the new dimension and 'stretches' it by N. # e.g. [0,1,2] -> [0,0,...,0,1,1,...,1,2,2,...,2] front = np.arange(d).repeat(N)[:,np.newaxis]

# This puts these two together. cur = np.column_stack((front,cur)) N *= d

return cur

Show replies by date

Sebastian Haase

14 Dec 14 Dec

4:31 a.m.

Do you know about N.fromiter() ?

-Sebastian Haase

On Dec 14, 2007 12:33 AM, Jonathan Taylor jonathan.taylor@utoronto.ca wrote:

...

I was needing an array representation of ndindex since ndindex only gives an iterator but array(list(ndindex)) takes too long. There is prob some obvious way to do this I am missing but if not feel free to include this code which is much faster.

In [252]: time a=np.array(list(np.ndindex(10,10,10,10,10,10))) CPU times: user 11.61 s, sys: 0.09 s, total: 11.70 s Wall time: 11.82

In [253]: time a=ndtuples(10,10,10,10,10,10) CPU times: user 0.32 s, sys: 0.21 s, total: 0.53 s Wall time: 0.60

def ndtuples(*dims): """Fast implementation of array(list(ndindex(*dims)))."""

# Need a list because we will go through it in reverse popping # off the size of the last dimension. dims = list(dims)

# N will keep track of the current length of the indices. N = dims.pop()

# At the beginning the current list of indices just ranges over the # last dimension. cur = np.arange(N) cur = cur[:,np.newaxis]

while dims != []:
   d = dims.pop()

   # This repeats the current set of indices d times.
   # e.g. [0,1,2] -> [0,1,2,0,1,2,...,0,1,2]
   cur = np.kron(np.ones((d,1)),cur)

   # This ranges over the new dimension and 'stretches' it by N.
   # e.g. [0,1,2] -> [0,0,...,0,1,1,...,1,2,2,...,2]
   front = np.arange(d).repeat(N)[:,np.newaxis]

   # This puts these two together.
   cur = np.column_stack((front,cur))
   N *= d
return cur _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Stefan van der Walt

8:32 a.m.

Hi Sebastian

N.fromiter only works on 1D arrays. I thought the following may work, but it doesn't:

np.fromiter(np.ndindex(10,10,10),N.dtype((int,3)))

This kind of loop is probably best implemented in C, although I think Jonathan's version is rather clever.

Regards Stéfan

On Fri, Dec 14, 2007 at 10:31:56AM +0100, Sebastian Haase wrote:

...

Do you know about N.fromiter() ?

-Sebastian Haase

On Dec 14, 2007 12:33 AM, Jonathan Taylor jonathan.taylor@utoronto.ca wrote:

...
I was needing an array representation of ndindex since ndindex only gives an iterator but array(list(ndindex)) takes too long. There is prob some obvious way to do this I am missing but if not feel free to include this code which is much faster.

In [252]: time a=np.array(list(np.ndindex(10,10,10,10,10,10))) CPU times: user 11.61 s, sys: 0.09 s, total: 11.70 s Wall time: 11.82

In [253]: time a=ndtuples(10,10,10,10,10,10) CPU times: user 0.32 s, sys: 0.21 s, total: 0.53 s Wall time: 0.60

def ndtuples(*dims): """Fast implementation of array(list(ndindex(*dims)))."""

# Need a list because we will go through it in reverse popping # off the size of the last dimension. dims = list(dims)

# N will keep track of the current length of the indices. N = dims.pop()

# At the beginning the current list of indices just ranges over the # last dimension. cur = np.arange(N) cur = cur[:,np.newaxis]

while dims != []:
   d = dims.pop()

   # This repeats the current set of indices d times.
   # e.g. [0,1,2] -> [0,1,2,0,1,2,...,0,1,2]
   cur = np.kron(np.ones((d,1)),cur)

   # This ranges over the new dimension and 'stretches' it by N.
   # e.g. [0,1,2] -> [0,0,...,0,1,1,...,1,2,2,...,2]
   front = np.arange(d).repeat(N)[:,np.newaxis]

   # This puts these two together.
   cur = np.column_stack((front,cur))
   N *= d
return cur

Jarrod Millman

18 Dec 18 Dec

1:59 p.m.

Hey Jonathan,

Thanks for providing this. I created a ticket for now: http://scipy.org/scipy/numpy/ticket/636

I will take some time to add this functionality to NumPy before the end of the year.

Thanks,

-- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

Jarrod Millman

22 Feb 22 Feb

6:26 p.m.

Could you provide more details about this to the ticket I created based on your email: http://projects.scipy.org/scipy/numpy/ticket/636

Thanks,

On Thu, Dec 13, 2007 at 3:33 PM, Jonathan Taylor jonathan.taylor@utoronto.ca wrote:

...

I was needing an array representation of ndindex since ndindex only gives an iterator but array(list(ndindex)) takes too long. There is prob some obvious way to do this I am missing but if not feel free to include this code which is much faster.

In [252]: time a=np.array(list(np.ndindex(10,10,10,10,10,10))) CPU times: user 11.61 s, sys: 0.09 s, total: 11.70 s Wall time: 11.82

In [253]: time a=ndtuples(10,10,10,10,10,10) CPU times: user 0.32 s, sys: 0.21 s, total: 0.53 s Wall time: 0.60

def ndtuples(*dims): """Fast implementation of array(list(ndindex(*dims)))."""
# Need a list because we will go through it in reverse popping
# off the size of the last dimension.
dims = list(dims)

# N will keep track of the current length of the indices.
N = dims.pop()

# At the beginning the current list of indices just ranges over the
# last dimension.
cur = np.arange(N)
cur = cur[:,np.newaxis]

while dims != []:

    d = dims.pop()

    # This repeats the current set of indices d times.
    # e.g. [0,1,2] -> [0,1,2,0,1,2,...,0,1,2]
    cur = np.kron(np.ones((d,1)),cur)

    # This ranges over the new dimension and 'stretches' it by N.
    # e.g. [0,1,2] -> [0,0,...,0,1,1,...,1,2,2,...,2]
    front = np.arange(d).repeat(N)[:,np.newaxis]

    # This puts these two together.
    cur = np.column_stack((front,cur))
    N *= d

return cur
Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

-- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

5587

Age (days ago)

5658

Last active (days ago)

List overview

Download

4 comments

4 participants

tags (0)

participants (4)

Jarrod Millman
Jonathan Taylor
Sebastian Haase
Stefan van der Walt