Interesting list() un-optimization
wolfgang.maier at biologie.uni-freiburg.de
Thu Mar 7 21:41:37 CET 2013
> >>> Iterators do not generally have __len__ methods.
> >>> >>> len(iter(range(10)))
> >>> Traceback (most recent call last):
> >>> File "<stdin>", line 1, in <module>
> >>> TypeError: object of type 'range_iterator' has no len()
> >> But iterators have a length hint method that are used for some
> >> optimizations and preallocations, too.
> >> >>> i = iter(range(10))
> >> >>> i.__length_hint__()
> >> 10
> >> See http://www.python.org/dev/peps/pep-0424/
very interesting (hadn't heard of it)! Just checked the PEP,
then tested list()'s behavior, and it is just as described:
print ('len() called')
print ('hint requested')
Traceback (most recent call last):
File "<pyshell#79>", line 1, in <module>
TypeError: an integer is required
[1, 2, 3]
so list() first tries to call the iterable's __len__ method. If that raises a
TypeError it falls back to __length_hint__ .
What I still don't know is how the listiterator object's __length_hint__ works.
Why, in this case, does it know that it has a length of 3 ? The PEP does not
provide any hint how a reasonable hint could be calculated.
> And how exactly would it do that, without either doing what __len__ does or
> reading the whole result set into memory?
a very good question.
More information about the Python-list