
Hi Folks I have this situation
from timeit import Timer reps = 5
t = Timer('itertools.combinations(range(1,10),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.59740447998e-05 t = Timer('itertools.combinations(range(1,100),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.74999237061e-05
t = Timer('list(itertools.combinations(range(1,10),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 5.31673431396e-05 t = Timer('list(itertools.combinations(range(1,100),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 0.0556231498718
You can see list(itertools.combinations(range(1,100),3)) is terrible!! If you change to range(1,100000) your computer will lock. So I would like to know a good way to convert <itertools.combinations object> to ndarray? fast! without use list Is it possible?
x = itertools.combinations(range(1,10),3) x <itertools.combinations object at 0x25f1520>
I tried this from http://docs.python.org/library/itertools.html?highlight=itertools#itertools....
numpy.fromiter(itertools.combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
and this from http://docs.python.org/library/itertools.html?highlight=itertools#itertools.... import numpy from itertools import * from numpy import * def combinations(iterable, r): pool = tuple(iterable) n = len(pool) for indices in permutations(range(n), r): if sorted(indices) == list(indices): yield tuple(pool[i] for i in indices) numpy.fromiter(combinations(range(1,10),3), int, count=-1)
numpy.fromiter(combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
I like itertools.combinations performance but I need convert it to numpy. Best Regards mario

On Fri, Dec 3, 2010 at 6:31 AM, Mario Moura <moura.mario@gmail.com> wrote:
Hi Folks
I have this situation
from timeit import Timer reps = 5
t = Timer('itertools.combinations(range(1,10),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.59740447998e-05 t = Timer('itertools.combinations(range(1,100),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.74999237061e-05
t = Timer('list(itertools.combinations(range(1,10),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 5.31673431396e-05 t = Timer('list(itertools.combinations(range(1,100),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 0.0556231498718
You can see list(itertools.combinations(range(1,100),3)) is terrible!!
If you change to range(1,100000) your computer will lock.
So I would like to know a good way to convert <itertools.combinations object> to ndarray? fast! without use list Is it possible?
x = itertools.combinations(range(1,10),3) x <itertools.combinations object at 0x25f1520>
I tried this from
http://docs.python.org/library/itertools.html?highlight=itertools#itertools....
numpy.fromiter(itertools.combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
and this from
http://docs.python.org/library/itertools.html?highlight=itertools#itertools....
import numpy from itertools import * from numpy import *
def combinations(iterable, r): pool = tuple(iterable) n = len(pool) for indices in permutations(range(n), r): if sorted(indices) == list(indices): yield tuple(pool[i] for i in indices)
numpy.fromiter(combinations(range(1,10),3), int, count=-1)
numpy.fromiter(combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
I like itertools.combinations performance but I need convert it to numpy.
The docstring for numpy.fromiter() says it creates a 1D array. You can use it with itertools.combinations if you specify a dtype for a 1D structured array. Here's an example (I'm using ipython with the -pylab option, so the numpy functions have all been imported): In [1]: from itertools import combinations In [2]: dt = dtype('i,i,i') In [3]: a = fromiter(combinations(range(100),3), dtype=dt, count=-1) In [4]: b = array(list(combinations(range(100),3))) In [5]: all(a.view(int).reshape(-1,3) == b) Out[5]: True In [6]: timeit a = fromiter(combinations(range(100),3), dtype=dt, count=-1) 10 loops, best of 3: 92.7 ms per loop In [7]: timeit b = array(list(combinations(range(100),3))) 1 loops, best of 3: 627 ms per loop In [8]: a[:3] Out[8]: array([(0, 1, 2), (0, 1, 3), (0, 1, 4)], dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<i4')]) In [9]: b[:3] Out[9]: array([[0, 1, 2], [0, 1, 3], [0, 1, 4]]) In the above example, 'a' is a 1D structured array; each element of 'a' holds one of the combinations. If you need it, you can create a 2D view with a.view(int).reshape(-1,3). Warren

Hi Mr. Weckesser Thanks a lot! Works fine! Regards Mario 2010/12/3 Warren Weckesser <warren.weckesser@enthought.com>:
On Fri, Dec 3, 2010 at 6:31 AM, Mario Moura <moura.mario@gmail.com> wrote:
Hi Folks
I have this situation
from timeit import Timer reps = 5
t = Timer('itertools.combinations(range(1,10),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.59740447998e-05 t = Timer('itertools.combinations(range(1,100),3)', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 1.74999237061e-05
t = Timer('list(itertools.combinations(range(1,10),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 5.31673431396e-05 t = Timer('list(itertools.combinations(range(1,100),3))', 'import itertools') print sum(t.repeat(repeat=reps, number=1)) / reps 0.0556231498718
You can see list(itertools.combinations(range(1,100),3)) is terrible!!
If you change to range(1,100000) your computer will lock.
So I would like to know a good way to convert <itertools.combinations object> to ndarray? fast! without use list Is it possible?
x = itertools.combinations(range(1,10),3) x <itertools.combinations object at 0x25f1520>
I tried this from
http://docs.python.org/library/itertools.html?highlight=itertools#itertools....
numpy.fromiter(itertools.combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
and this from
http://docs.python.org/library/itertools.html?highlight=itertools#itertools....
import numpy from itertools import * from numpy import *
def combinations(iterable, r): pool = tuple(iterable) n = len(pool) for indices in permutations(range(n), r): if sorted(indices) == list(indices): yield tuple(pool[i] for i in indices)
numpy.fromiter(combinations(range(1,10),3), int, count=-1)
numpy.fromiter(combinations(range(1,10),3), int, count=-1) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: setting an array element with a sequence.
I like itertools.combinations performance but I need convert it to numpy.
The docstring for numpy.fromiter() says it creates a 1D array. You can use it with itertools.combinations if you specify a dtype for a 1D structured array. Here's an example (I'm using ipython with the -pylab option, so the numpy functions have all been imported):
In [1]: from itertools import combinations
In [2]: dt = dtype('i,i,i')
In [3]: a = fromiter(combinations(range(100),3), dtype=dt, count=-1)
In [4]: b = array(list(combinations(range(100),3)))
In [5]: all(a.view(int).reshape(-1,3) == b) Out[5]: True
In [6]: timeit a = fromiter(combinations(range(100),3), dtype=dt, count=-1) 10 loops, best of 3: 92.7 ms per loop
In [7]: timeit b = array(list(combinations(range(100),3))) 1 loops, best of 3: 627 ms per loop
In [8]: a[:3] Out[8]: array([(0, 1, 2), (0, 1, 3), (0, 1, 4)], dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<i4')])
In [9]: b[:3] Out[9]: array([[0, 1, 2], [0, 1, 3], [0, 1, 4]])
In the above example, 'a' is a 1D structured array; each element of 'a' holds one of the combinations. If you need it, you can create a 2D view with a.view(int).reshape(-1,3).
Warren
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (2)
-
Mario Moura
-
Warren Weckesser