On Apr 27, 2020, at 13:41, Christopher Barker <pythonchb@gmail.com> wrote:
SIDE NOTE: this is reminding me that there have been calls in the past for an optional __len__ protocol for iterators that are not proper sequences, but DO know their length -- maybe one more place to use that if it existed.
But __len__ doesn’t really make sense on iterators. And no iterator is a proper sequence, so I think you meant _iterables_ that aren’t proper sequences anyway—and that’s already there: xs = {1, 2, 3} len(xs) # 3 isinstance(xs, collections.abc.Sized) # True I think the issue is that people don’t actually want zip to be an Iterator, they want it to be a smarter Iterable that preserves (at least) Sized from its inputs. The same way, e.g., dict.items or memoryview does. The same way range is lazy but not an Iterator. And it’s not just zip; the same thing is true for map, enumerate, islice, etc. And it’s also not just Sized. It would be just as cool if zip, enumerate, etc. preserved Reversible. In fact, “how do I both enumerate and reverse” comes up often enough that I’ve got a reverse_enumerate function in my toolbox to work around it. And, for that matter, why do they have to be only one-shot-iterable unless their input is? Again, dict.items and range come to mind, and there’s no real reason zip, map, islice, etc. couldn’t preserve as much of their input behavior as possible: xs = [1, 2, 3] ys = map(lambda x: x*3, xs) len(ys) # 3 reversed(enumerate(ys))[-1] # (0, 3) Of course it’s not always possible to preserve all behavior: xs = [1, 2, 3] ys = filter(lambda x: x%2, xs) len(ys) # still a TypeError even though xs is sized … but the cases where it is or isn’t possible can all be worked out for each function and each ABC: filter can _never_ preserve Sized but can _always_ preserves Reversible, etc. This is clearly feasible—Swift does it, and C++ is trying to do it in their next version, and Python already does it in a few special cases (as mentioned earlier), just not in all (or even most) of the potentially useful cases. The only really hard part of this is designing a framework that makes it possible to write all those views simply. You don’t want to have to write five different map view classes for all the ways a map can act based on its inputs, and then repeat 80% of that same work again for filter, and again for islice and so on. The boilerplate would be insane. (See the Swift 1.0 stdlib for an example of how horrible it could be, and they only implemented a handful of the possibilities.) And, except for a couple of things (notably genexprs), most of this could be written as a third-party library today. (And if it existed and people were using it widely, it would be pretty easy to argue that it should come with Python, so that it _could_ handle those last few things like genexprs, and also to serve as an example to encourage third-party libraries like toolz to similarly implement smart views instead of dumb iterators, and also as helpers to make that easier for them. That argument might or might not win the day, but at least it’s obvious what it would look like.) So I suspect the only reason nobody’s done so is that you don’t actually run into a need for it very often. How often do you actually need the result of zip to be Sized anyway? At least for me, it’s not very often. Whenever I run into any of these needs, I start thinking about the fully general solution, but put it off until I run into a second good use for it and meanwhile write a simple 2-minute workaround for my immediate use (or add a new special case like reversed_enumerate to my toolbox), and then by the time I run into another need for it, it’s been so long that I’ve almost forgotten the idea… But maybe there would be a lot more demand for this if people knew the idea was feasible? Maybe there are people who have tons of real-life examples where they could use a Sized zip or a Reversible enumerate or a Sequence map, and they just never thought they could have it so they never tried or asked?