# Consume an iterable

Peter Otten __peter__ at web.de
Sat Jan 23 13:45:43 CET 2010

```Muhammad Alkarouri wrote:

> Thanks everyone, but not on my machine (Python 2.6.1, OS X 10.6) it's
> not:
>
>
> In [1]: from itertools import count, islice
>
> In [2]: from collections import deque
>
> In [3]: i1=count()
>
> In [4]: def consume1(iterator, n):
>    ...:     deque(islice(iterator, n), maxlen=0)
>    ...:
>    ...:
>
> In [5]: i2=count()
>
> In [6]: def consume2(iterator, n):
>    ...:     for _ in islice(iterator, n): pass
>    ...:
>    ...:
>
> In [7]: timeit consume1(i1, 10)
> 1000000 loops, best of 3: 1.63 us per loop
>
> In [8]: timeit consume2(i2, 10)
> 1000000 loops, best of 3: 846 ns per loop
>
>
> Can somebody please test whether it is only my machine or is this
> reproducible?

I can reproduce it. The deque-based approach has a bigger constant overhead
but better per-item performance. Its asymptotical behaviour is therefore
better.

\$ python consume_timeit.py
consume_deque
10: 1.77500414848
100: 3.73333001137
1000: 24.7235469818

consume_forloop
10: 1.22008490562
100: 5.86271500587
1000: 52.2449371815

consume_islice
10: 0.897439956665
100: 1.51542806625
1000: 7.70061397552

\$ cat consume_timeit.py
from collections import deque
from itertools import islice, repeat

def consume_deque(n, items):
deque(islice(items, n), maxlen=0)

def consume_forloop(n, items):
for _ in islice(items, n):
pass

def consume_islice(n, items):
next(islice(items, n-1, None), None)

def check(fs):
for consume in fs:
items = iter(range(10))
consume(3, items)
rest = list(items)
assert rest == range(3, 10), consume.__name__

if __name__ == "__main__":
fs = consume_deque, consume_forloop, consume_islice
check(fs)

items = repeat(None)

from timeit import Timer
for consume in fs:
print consume.__name__
for n in (10, 100, 1000):
print "%6d:" % n,
print Timer("consume(%s, items)" % n,
"from __main__ import consume, items").timeit()
print
\$

With next(islice(...), None) I seem to have found a variant that beats both
competitors.

Peter

```