Before jumping in: In many cases, when you want to reverse an enumerate, it’s small and fixed-sized, so there’s a trivial way to do this: Just store the enumerate iterator in a tuple, and tuples are reversible. for idx, value in reversed(tuple(enumerate(stuff))): But of course there are some cases where this isn’t appropriate, like enumerating a fixed-size but huge input.

On Apr 1, 2020, at 19:23, Steven D'Aprano wrote:

[Ilya]

...

I needed reversed(enumerate(x: list)) in my code, and have discovered that it wound't work. This is disappointing because operation is well defined.

It isn't really well-defined, since enumerate can operate on infinite iterators, and you cannot reverse an infinite stream.

...

However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too.

Agreed—but this is just a small piece of a much wider issue. Today, enumerate is always an Iterator. It’s never reversible. But it’s also not sized, or subscriptable, or in-testable, even if you give it inputs that are. And it’s not just enumerate—the same is true for map, filter, zip, itertools.islice, itertools.dropwhile, etc. There’s no reason these things couldn’t all be views, just like the existing dict views (and other things like memoryview and third-party things like numpy array slices). In fact, they already are in Swift, and will be in C++20.

My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense.

Actually, that doesn’t work—it has to be Sized as well. More generally, it’s rarely _quite_ as simple as just “views support the same operations as the things they view”. An enumerate can be a Sequence if its input is, but a filter can’t. A map with multiple inputs isn’t Reversible unless they’re all not just Reversible but Sized, although a map with only one input doesn’t need it to be Sized. And so on. But none of these things are hard, it’s just a bunch of work to go through all the input types for all the view types and write up the rules. (Or steal them from another language or library that already did that work…)

But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard.

Agreed. I don’t think we need to wait until someone designs and writes a complete viewtools library and submits it for stdlib inclusion before we can consider adding just one extension to one iterator. But I do think we want to add the one(s) that are most useful if any, not just whichever ones people think of first. I’ve personally wanted to reverse a map or a filter more often than an enumerate, but examples would easily convince me that that’s just me, and reversing enumerate is more needed.

One potentially serious question: what should `enumerate.__reversed__` do when given a starting value?

reversed(enumerate('abc', 1))

I don’t think this is a problem. When you reversed(tuple(enumerate('abc', 1))) today, what do you get? You presumably don’t even need to look that up or try it out. It would be pretty confusing if it were different without the tuple.

I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis. In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful. Best, Neil On Thursday, January 21, 2021 at 6:23:20 AM UTC-5 Nuri Jung wrote:

So, what is the conclusion? I also think reversed(enumerate(some_seq)) will be very useful in many cases. It should: 1) work the same as reversed(tuple(enumerate(...))) for "reversible" objects as argument of enumerate, 2) raise TypeError if the object is not reversible.

Or, another option would be adding a "step" keyword argument to enumerate. Then, reversing enumerate would be much easier, like this: enumerate(reversed(some_seq), step=-1) _______________________________________________ Python-ideas mailing list -- python...@python.org To unsubscribe send an email to python-id...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python...@python.org/message/BW2XKRZ4T... https://mail.python.org/archives/list/python-ideas@python.org/message/BW2XKR... Code of Conduct: http://python.org/psf/codeofconduct/

On Thu, Jan 21, 2021 at 1:26 PM Chris Angelico wrote:

On Fri, Jan 22, 2021 at 5:21 AM Neil Girdhar wrote:

...

I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences. I think it's good idea, but it adds complexity and work, which I guess needs to be justified on a case-by-case basis.

In short, this has nothing to do with reversed. If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34], which could also be useful. I think it makes Python slightly more perfect and more beautiful.

list(enumerate(some_list)) will give you a sequence, if that's what you want.

Right. And reversed(list(enumerate(some_list)) also works. The point is that if you return sequence views (as Andrew mentioned), you don't need to cast to list explicitly, and you don't pay the computational cost of that either.

ChrisA _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WW3JHS... Code of Conduct: http://python.org/psf/codeofconduct/

--

--- You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/3wGART2ECXQ/unsubscribe. To unsubscribe from this group and all its topics, send an email to python-ideas+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/python-ideas/CAPTjJmpPchBQEWVP1Mt8UW3aaquT....

On Thu, Jan 21, 2021 at 10:43 AM Neil Girdhar wrote:

...

...

I've seen this proposed here before. The general idea is that some iterator transformations (like enumerate) should return sequences when they're applied to sequences.

I'm not so sure -- I don't think I want a sequence returned. In fact, in this case, just the opposite. If you wanted a sequence, you could simply do: reversed(list(enumerate(the_iterable))) Rather, you want (or I want, anyway) is for various iterators to be able to do things that require a Sequence, when given a Sequence. Or, maybe not even a full Sequence, but rather, some particular aspect of a sequence -- like the length, or indexability. enumerate() with a reversed flag is one. Another, I'd like is islice to support negative indexing for indexing from the end. There are probably many others. I'd love to see these supported. For this example: enumerate(an_iterable, reversed=True) would "just work" if an_iterable had a __len__ and a __getitem__. It would raise a TypeError if passed a non-lengthed, or non-indexable iterable. -CHB

...

In short, this has nothing to do with reversed.

I agree.

...

If you made enumerate return a sequence when its input is a sequence, you would also be able to do enumerate(some_list)[34],

As Chris A mentioned, it's quite easy to wrap list() or tuple() around it if you want that. - Chris B -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Fri, Jan 22, 2021 at 8:31 AM Neil Girdhar wrote:

It's possible for reversed(enumerate(...)) to just work if enumerate of a sequence were to return a sequence view. Then you would also get all the other sequence operations for free like enumerate(...)[23:27], len(enumerate(...)), etc.

That would break backward compatibility. The behaviour of enumerate() at the moment isn't compatible with it just suddenly being a sequence. It would have to have some way to be told "please return a sequence". We currently have a way to say "please return an eager sequence", so the only thing we don't easily have is "please return a lazy sequence", and that's possible with about four lines of custom class. Note that slicing is NOT easy. The proposed semantics for a reversed enumeration would make slicing extremely odd.

...

As Chris A mentioned, it's quite easy to wrap list() or tuple() around it if you want that.

Yes, you can alternatively wrap with list and pay the computational cost. However, as was suggested by Andrew in this thread, but has come up many times before, the native iteration tools could return views. This saves the user having to build her own custom classes (along with tests, etc.). I happen to think this is simpler and more beautiful.

It wouldn't work. With enumerate, you could get away with returning a view, but let's look at map. Should that return a view? If so, does map(...)[4] cache the result or not? If it does, you're paying the memory cost of (potentially) the full eager list, in which case you may as well just make the list. If it doesn't, you break expectations badly by having it call the function more than once, potentially giving different results. Either way, it's far better to write your own, giving you the freedom to define the semantics the way you want to. It's even worse with filter(). It can't possibly know which is the 5th element without counting from the start ("hmm, that's Fire... Earth... Water... Air... Boron. Ah! Boron!"). What about itertools.tee()? I think the whole point of that is to hybridize laziness and caching, but if you're going to give it a sequence, you may as well just get multiple iterators. I'm eyeballing itertools and really not seeing any that would benefit from magically returning sequences. Several of them specifically exist to provide sequence-like behaviour on non-sequence iterables (eg islice), and others don't have a direct correlation between input index and output index (eg dropwhile) or have other discrepancies (eg takewhile). Rather than try to propose that ALL native iteration tools return views, find one that would actually benefit, and propose that. Unfortunately, I think you'll run into the same problem that it's no longer backward compatible, so it's still going to need a spelling to request that it behave in sequence view mode, which is probably best written as something like "import seqview; seqview.map(...)". ChrisA

On Fri, Jan 22, 2021 at 11:10 AM Random832 wrote:

On Thu, Jan 21, 2021, at 18:48, Chris Angelico wrote:

...

Note that slicing is NOT easy. The proposed semantics for a reversed enumeration would make slicing extremely odd.

What proposed semantics? You were the one who posted a pure-python implementation that didn't bother to implement slicing.

Ah, my bad. This thread started out on python-dev before migrating to -ideas, and I believe the proposed semantics may have been on one of the early -dev posts. The proposal was, effectively, to be the same as enumerating, making a concrete list, and then reversing (only, without actually building the list, of course). So enumerate("abc") yields [(0,'a'),(1,'b'), (2,'c')], and iterating backwards should yield those same tuples in reverse order. But if you take the reversed enumeration, iterate over it a bit, and then slice it, the semantics will be extremely bizarre.

It's easy enough to add slicing to your Enumerated class concept, though.

class Enumerated: def __init__(self, basis, indices=None): self.basis = basis self.indices = indices if indices is not None else range(len(basis)) def __getitem__(self, idx): if isinstance(idx, slice): return Enumerated(self.basis, indices=self.indices[idx]) else: return (self.indices[idx], self.basis[self.indices[idx]]) def __len__(self): return len(self.indices)

Yeah, I don't think that'll work if you slice more than once, especially with some iteration in between. The precise semantics for mixing iteration and slicing are complex enough and variable enough that you may as well just go for a concrete list at that point. ChrisA

On Fri, Jan 22, 2021 at 11:48 AM Neil Girdhar wrote:

If enumerate(some_sequence) returns a sequence view, iterating over that sequence view does not advance it—just like how DictViews are not altered by iteration. Same thing if reversed(some_sequence) returns a sequence view.

Then that's backward incompatible with current behaviour, and cannot be done just as "it's a sequence so return a sequence view". Hence the suggestion of a parameter to control this - or, my preferred way: a "seqview" module from which you can import a sequence-view-only version of various iterator functions. ChrisA