Re: [Numpy-discussion] itertools.combinations to numpy

Hi Mr. Weckesser Thanks a lot! Works fine! Regards Mario 2010/12/3 Warren Weckesser :

On Fri, Dec 3, 2010 at 6:31 AM, Mario Moura wrote:

...

Hi Folks

I have this situation

...

...

...

from timeit import Timer reps = 5

t = Timer('itertools.combinations(range(1,10),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.59740447998e-05 t = Timer('itertools.combinations(range(1,100),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.74999237061e-05

t = Timer('list(itertools.combinations(range(1,10),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 5.31673431396e-05 t = Timer('list(itertools.combinations(range(1,100),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 0.0556231498718

You can see list(itertools.combinations(range(1,100),3)) is terrible!!

If you change to range(1,100000) your computer will lock.

So I would like to know a good way to convert to ndarray? fast! without use list Is it possible?

...

...

...

x = itertools.combinations(range(1,10),3) x

I tried this from

http://docs.python.org/library/itertools.html?highlight=itertools#itertools....

...

...

...

numpy.fromiter(itertools.combinations(range(1,10),3), int, count=-1) Traceback (most recent call last):  File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.

and this from

http://docs.python.org/library/itertools.html?highlight=itertools#itertools....

import numpy from itertools import * from numpy import *

def combinations(iterable, r):    pool = tuple(iterable)    n = len(pool)    for indices in permutations(range(n), r):        if sorted(indices) == list(indices):            yield tuple(pool[i] for i in indices)

numpy.fromiter(combinations(range(1,10),3), int, count=-1)

...

...

...

numpy.fromiter(combinations(range(1,10),3), int, count=-1) Traceback (most recent call last):  File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.

I like itertools.combinations performance but I need convert it to numpy.

The docstring for numpy.fromiter() says it creates a 1D array.  You can use it with itertools.combinations if you specify a dtype for a 1D  structured array.  Here's an example (I'm using ipython with the -pylab option, so the numpy functions have all been imported):

In [1]: from itertools import combinations

In [2]: dt = dtype('i,i,i')

In [3]: a = fromiter(combinations(range(100),3), dtype=dt, count=-1)

In [4]: b = array(list(combinations(range(100),3)))

In [5]: all(a.view(int).reshape(-1,3) == b) Out[5]: True

In [6]: timeit a = fromiter(combinations(range(100),3), dtype=dt, count=-1) 10 loops, best of 3: 92.7 ms per loop

In [7]: timeit b = array(list(combinations(range(100),3))) 1 loops, best of 3: 627 ms per loop

In [8]: a[:3] Out[8]: array([(0, 1, 2), (0, 1, 3), (0, 1, 4)],       dtype=[('f0', '

In [9]: b[:3] Out[9]: array([[0, 1, 2],        [0, 1, 3],        [0, 1, 4]])

In the above example, 'a' is a 1D structured array; each element of 'a' holds one of the combinations.  If you need it, you can create a 2D view with a.view(int).reshape(-1,3).

Warren

_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion