[Python-Dev] PEP 0424: A method for exposing a length hint

[M Stefan]

On 7/16/2012 9:54 AM, Stefan Behnel wrote:
> Mark Shannon, 15.07.2012 16:14:
>> Alex Gaynor wrote:
>>> CPython currently defines an ``__length_hint__`` method on several types,
>>> such
>>> as various iterators. This method is then used by various other functions
>>> (such as ``map``) to presize lists based on the estimated returned by
>> Don't use "map" as an example.
>> map returns an iterator so it doesn't need __length_hint__
> Right. It's a good example for something else, though. As I mentioned
> before, iterators should be able to propagate the length hint of an
> underlying iterator, e.g. in generator expressions or map(). I consider
> that an important feature that the protocol must support.
>
> Stefan
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mstefanro%40gmail.com
>
map() is quite problematic in this matter, and may actually
benefit from the existence of __length_hint__.
It is very easy to create an infinite loop currently by doing stuff like 
x=[1]; x+=map(str,x)
[61081 refs]
 >>> x=[1]; x+=map(str,x)
Traceback (most recent call last):
   ...
MemoryError
[120959834 refs]
 >>> len(x)
120898752

Obviously, this won't cause an infinite loop in Python2 where map is 
non-lazy.
Also, this won't work for all mutable containers, because not all of 
them permit
adding elements while iterating:
 >>> s=set([1]); s.update(map(str,s))
Traceback (most recent call last):
   ...
RuntimeError: Set changed size during iteration
[61101 refs]
 >>> s
{1, '1'}
[61101 refs]
 >>> del s
[61099 refs]

If map objects were to disallow changing the size of the container
while iterating (I can't really think of an use-case in which such a
limitation would be harmful), it might as well be with __length_hint__.

Also, what would iter([1,2,3]).__length_hint__() return? 3 or unknown?
If 3, then the semantics of l=[1,2,3]; l += iter(l) will change 
(infinite loop
without __length_hint__ vs. list of 6 elements with __length_hint__).
If unknown, then it doesn't seem like there are very many places where
__length_hint__ can return anything but unknown.

Regards,
   Stefan M