[Numpy-discussion] itertools.combinations to numpy
Warren Weckesser
warren.weckesser at enthought.com
Fri Dec 3 09:50:57 EST 2010
On Fri, Dec 3, 2010 at 6:31 AM, Mario Moura <moura.mario at gmail.com> wrote:
> Hi Folks
>
> I have this situation
>
> >>> from timeit import Timer
> >>> reps = 5
> >>>
> >>> t = Timer('itertools.combinations(range(1,10),3)', 'import itertools')
> >>> print sum(t.repeat(repeat=reps, number=1)) / reps
> 1.59740447998e-05
> >>> t = Timer('itertools.combinations(range(1,100),3)', 'import itertools')
> >>> print sum(t.repeat(repeat=reps, number=1)) / reps
> 1.74999237061e-05
> >>>
> >>> t = Timer('list(itertools.combinations(range(1,10),3))', 'import
> itertools')
> >>> print sum(t.repeat(repeat=reps, number=1)) / reps
> 5.31673431396e-05
> >>> t = Timer('list(itertools.combinations(range(1,100),3))', 'import
> itertools')
> >>> print sum(t.repeat(repeat=reps, number=1)) / reps
> 0.0556231498718
> >>>
>
> You can see list(itertools.combinations(range(1,100),3)) is terrible!!
>
> If you change to range(1,100000) your computer will lock.
>
> So I would like to know a good way to convert <itertools.combinations
> object> to ndarray? fast! without use list
> Is it possible?
>
> >>> x = itertools.combinations(range(1,10),3)
> >>> x
> <itertools.combinations object at 0x25f1520>
> >>>
>
> I tried this from
>
> http://docs.python.org/library/itertools.html?highlight=itertools#itertools.combinations
>
> >>> numpy.fromiter(itertools.combinations(range(1,10),3), int, count=-1)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ValueError: setting an array element with a sequence.
> >>>
>
> and this from
>
> http://docs.python.org/library/itertools.html?highlight=itertools#itertools.combinations
>
> import numpy
> from itertools import *
> from numpy import *
>
> def combinations(iterable, r):
> pool = tuple(iterable)
> n = len(pool)
> for indices in permutations(range(n), r):
> if sorted(indices) == list(indices):
> yield tuple(pool[i] for i in indices)
>
>
> numpy.fromiter(combinations(range(1,10),3), int, count=-1)
>
> >>> numpy.fromiter(combinations(range(1,10),3), int, count=-1)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> ValueError: setting an array element with a sequence.
> >>>
>
>
> I like itertools.combinations performance but I need convert it to numpy.
>
>
The docstring for numpy.fromiter() says it creates a 1D array. You can use
it with itertools.combinations if you specify a dtype for a 1D structured
array. Here's an example (I'm using ipython with the -pylab option, so the
numpy functions have all been imported):
In [1]: from itertools import combinations
In [2]: dt = dtype('i,i,i')
In [3]: a = fromiter(combinations(range(100),3), dtype=dt, count=-1)
In [4]: b = array(list(combinations(range(100),3)))
In [5]: all(a.view(int).reshape(-1,3) == b)
Out[5]: True
In [6]: timeit a = fromiter(combinations(range(100),3), dtype=dt, count=-1)
10 loops, best of 3: 92.7 ms per loop
In [7]: timeit b = array(list(combinations(range(100),3)))
1 loops, best of 3: 627 ms per loop
In [8]: a[:3]
Out[8]:
array([(0, 1, 2), (0, 1, 3), (0, 1, 4)],
dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<i4')])
In [9]: b[:3]
Out[9]:
array([[0, 1, 2],
[0, 1, 3],
[0, 1, 4]])
In the above example, 'a' is a 1D structured array; each element of 'a'
holds one of the combinations. If you need it, you can create a 2D view
with a.view(int).reshape(-1,3).
Warren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101203/183b122b/attachment.html>
More information about the NumPy-Discussion
mailing list