[Python-ideas] __len__() for map()

[Steven D'Aprano]

On Wed, Nov 28, 2018 at 04:14:24PM +0100, E. Madison Bray wrote:

> For example, some function that used to expect some finite-sized
> sequence such as a list or tuple is now passed a "map", possibly
> wrapping one or more iterable of arbitrary, possibly non-finite size.
> For the purposes of some algorithm I have this is not useful and I
> need to convert it to a sequence anyways but don't want to do that
> without some guarantee that I won't blow up the user's memory usage.
> So I might want to check:
> 
> finite_definite = True
> for it in my_map.iters:
>     try:
>         len(it)
>     except TypeError:
>         finite_definite = False
> 
> if finite_definite:
>     my_seq = list(my_map)
> else:
>     # some other algorithm

But surely you didn't need to do this just because of *map*. Users could 
have passed an infinite, unsized iterable going back to Python 1 days 
with the old sequence protocol. They certainly could pass a generator or 
other opaque iterator apart from map. So I'm having trouble seeing why 
the Python 2/3 change to map made things worse for SageMath.

But in any case, this example comes back to the question of len again, 
and we've already covered why this is problematic. In case you missed 
it, let's take a toy example which demonstrates the problem:


def mean(it):
    if isinstance(it, map):
        # Hypothetical attribute access to the underlying iterable.
        n = len(it.iterable)  
        return sum(it)/n


Now let's pass a map object to it:

data = [1, 2, 3, 4, 5]
it = map(lambda x: x, data)
assert len(it.iterable) == 5
next(it); next(it); next(it)

assert mean(it) == 4.5
# fails, as it actually returns 9/5 instead of 9/2


-- 
Steve