
Hi, I needed reversed(enumerate(x: list)) in my code, and have discovered that it wound't work. This is disappointing because operation is well defined. It is also well defined for str type, range, and - in principle, but not yet in practice - on dictionary iterators - keys(), values(), items() as dictionaries are ordered now. It would also be well defined on any user type implementing __iter__, __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors. That's plenty of usecases, therefore I guess it would be quite useful to avoid hacky / inefficient solutions like described here: https://code.activestate.com/lists/python-list/706205/. If deemed useful, I would be interested in implementing this, maybe together with __reversed__ on dict keys, values, items. Best Regards, -- Ilya Kamen ----------- p.s. *Sketch* of what I am proposing: class reversible_enumerate: def __init__(self, iterable): self.iterable = iterable self.ctr = 0 def __iter__(self): for e in self.iterable: yield self.ctr, e self.ctr += 1 def __reversed__(self): try: ri = reversed(self.iterable) except Exception as e: raise Exception( "enumerate can only be reversed if iterable to enumerate can be reversed and has defined length." ) from e try: l = len(self.iterable) except Exception as e: raise Exception( "enumerate can only be reversed if iterable to enumerate can be reversed and has defined length." ) from e indexes = range(l-1, -1, -1) for i, e in zip(indexes, ri): yield i, e for i, c in reversed(reversible_enumerate("Hello World")): print(i, c) for i, c in reversed(reversible_enumerate([11, 22, 33])): print(i, c)

01.04.20 21:45, Ilya Kamenshchikov пише:
I am not sure that it is well defined. It was not clear to me how you define it until I looked at your reference implementation, but your implementation is not compatible with enumerate(). For example it fails test for next(enumerate('a')).
Could you please provide evidence that this feature would be quite useful? How much usecases can you find in the stdlib?

Hi Ilya, I'm not sure that this mailing list (Python-Dev) is the right place for this discussion, I think that Python-Ideas (CCed) is the correct place. For the benefit of Python-Ideas, I have left your entire post below, to establish context. [Ilya]
It isn't really well-defined, since enumerate can operate on infinite iterators, and you cannot reverse an infinite stream. Consider: def values(): while True: yield random.random() a, b = reversed(enumerate(values()) What should the first pair of (a, b) be? However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too. My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense. But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard. On the other hand, I think that there is a fairly lightweight work around. Define a helper function: def countdown(n): while True: yield n n -= 1 then call it like this: # reversed(enumerate(seq)) zip(countdown(len(seq)-1), reversed(seq))) So it isn't terribly hard to work around this. But I agree that it would be nice if enumerate encapsulated this for the caller. One potentially serious question: what should `enumerate.__reversed__` do when given a starting value? reversed(enumerate('abc', 1)) Should that yield...? # treat the start value as a start value (1, 'c'), (0, 'b'), (-1, 'a') # treat the start value as an end value (3, 'c'), (2, 'b'), (1, 'a') Something else? My preference would be to treat the starting value as an ending value. Steven On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:

Before jumping in: In many cases, when you want to reverse an enumerate, it’s small and fixed-sized, so there’s a trivial way to do this: Just store the enumerate iterator in a tuple, and tuples are reversible. for idx, value in reversed(tuple(enumerate(stuff))): But of course there are some cases where this isn’t appropriate, like enumerating a fixed-size but huge input.
...
Agreed—but this is just a small piece of a much wider issue. Today, enumerate is always an Iterator. It’s never reversible. But it’s also not sized, or subscriptable, or in-testable, even if you give it inputs that are. And it’s not just enumerate—the same is true for map, filter, zip, itertools.islice, itertools.dropwhile, etc. There’s no reason these things couldn’t all be views, just like the existing dict views (and other things like memoryview and third-party things like numpy array slices). In fact, they already are in Swift, and will be in C++20.
Actually, that doesn’t work—it has to be Sized as well. More generally, it’s rarely _quite_ as simple as just “views support the same operations as the things they view”. An enumerate can be a Sequence if its input is, but a filter can’t. A map with multiple inputs isn’t Reversible unless they’re all not just Reversible but Sized, although a map with only one input doesn’t need it to be Sized. And so on. But none of these things are hard, it’s just a bunch of work to go through all the input types for all the view types and write up the rules. (Or steal them from another language or library that already did that work…)
Agreed. I don’t think we need to wait until someone designs and writes a complete viewtools library and submits it for stdlib inclusion before we can consider adding just one extension to one iterator. But I do think we want to add the one(s) that are most useful if any, not just whichever ones people think of first. I’ve personally wanted to reverse a map or a filter more often than an enumerate, but examples would easily convince me that that’s just me, and reversing enumerate is more needed.
I don’t think this is a problem. When you reversed(tuple(enumerate('abc', 1))) today, what do you get? You presumably don’t even need to look that up or try it out. It would be pretty confusing if it were different without the tuple.

Ilya Kamenshchikov writes:
I needed reversed(enumerate(x: list)) in my code,
def reversed_enumerated_list(l): return zip(reversed(range(len(l)), reversed(l))
and have discovered that it wound't work.
I wouldn't have expected it to, because enumerate's result is unbounded. But I'm definitely warming to the idea that this should work for iterables with known length. Presumably the same implementation method used for reversed itself would work; not sure if that is what you proposed. The method used to implement reversibility should probably be exposed so that other code that wants a finite enumeration can determine what enumerate has discovered about the underlying iterable.

01.04.20 21:45, Ilya Kamenshchikov пише:
I am not sure that it is well defined. It was not clear to me how you define it until I looked at your reference implementation, but your implementation is not compatible with enumerate(). For example it fails test for next(enumerate('a')).
Could you please provide evidence that this feature would be quite useful? How much usecases can you find in the stdlib?

Hi Ilya, I'm not sure that this mailing list (Python-Dev) is the right place for this discussion, I think that Python-Ideas (CCed) is the correct place. For the benefit of Python-Ideas, I have left your entire post below, to establish context. [Ilya]
It isn't really well-defined, since enumerate can operate on infinite iterators, and you cannot reverse an infinite stream. Consider: def values(): while True: yield random.random() a, b = reversed(enumerate(values()) What should the first pair of (a, b) be? However, having said that, I think that your idea is not unreasonable. `enumerate(it)` in the most general case isn't reversable, but if `it` is reversable and sized, there's no reason why `enumerate(it)` shouldn't be too. My personal opinion is that this is a fairly obvious and straightforward enhancement, one which (hopefully!) shouldn't require much, if any, debate. I don't think we need a new class for this, I think enhancing enumerate to be reversable if its underlying iterator is reversable makes good sense. But if you can show some concrete use-cases, especially one or two from the standard library, that would help your case. Or some other languages which offer this functionality as standard. On the other hand, I think that there is a fairly lightweight work around. Define a helper function: def countdown(n): while True: yield n n -= 1 then call it like this: # reversed(enumerate(seq)) zip(countdown(len(seq)-1), reversed(seq))) So it isn't terribly hard to work around this. But I agree that it would be nice if enumerate encapsulated this for the caller. One potentially serious question: what should `enumerate.__reversed__` do when given a starting value? reversed(enumerate('abc', 1)) Should that yield...? # treat the start value as a start value (1, 'c'), (0, 'b'), (-1, 'a') # treat the start value as an end value (3, 'c'), (2, 'b'), (1, 'a') Something else? My preference would be to treat the starting value as an ending value. Steven On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:

Before jumping in: In many cases, when you want to reverse an enumerate, it’s small and fixed-sized, so there’s a trivial way to do this: Just store the enumerate iterator in a tuple, and tuples are reversible. for idx, value in reversed(tuple(enumerate(stuff))): But of course there are some cases where this isn’t appropriate, like enumerating a fixed-size but huge input.
...
Agreed—but this is just a small piece of a much wider issue. Today, enumerate is always an Iterator. It’s never reversible. But it’s also not sized, or subscriptable, or in-testable, even if you give it inputs that are. And it’s not just enumerate—the same is true for map, filter, zip, itertools.islice, itertools.dropwhile, etc. There’s no reason these things couldn’t all be views, just like the existing dict views (and other things like memoryview and third-party things like numpy array slices). In fact, they already are in Swift, and will be in C++20.
Actually, that doesn’t work—it has to be Sized as well. More generally, it’s rarely _quite_ as simple as just “views support the same operations as the things they view”. An enumerate can be a Sequence if its input is, but a filter can’t. A map with multiple inputs isn’t Reversible unless they’re all not just Reversible but Sized, although a map with only one input doesn’t need it to be Sized. And so on. But none of these things are hard, it’s just a bunch of work to go through all the input types for all the view types and write up the rules. (Or steal them from another language or library that already did that work…)
Agreed. I don’t think we need to wait until someone designs and writes a complete viewtools library and submits it for stdlib inclusion before we can consider adding just one extension to one iterator. But I do think we want to add the one(s) that are most useful if any, not just whichever ones people think of first. I’ve personally wanted to reverse a map or a filter more often than an enumerate, but examples would easily convince me that that’s just me, and reversing enumerate is more needed.
I don’t think this is a problem. When you reversed(tuple(enumerate('abc', 1))) today, what do you get? You presumably don’t even need to look that up or try it out. It would be pretty confusing if it were different without the tuple.

Ilya Kamenshchikov writes:
I needed reversed(enumerate(x: list)) in my code,
def reversed_enumerated_list(l): return zip(reversed(range(len(l)), reversed(l))
and have discovered that it wound't work.
I wouldn't have expected it to, because enumerate's result is unbounded. But I'm definitely warming to the idea that this should work for iterables with known length. Presumably the same implementation method used for reversed itself would work; not sure if that is what you proposed. The method used to implement reversibility should probably be exposed so that other code that wants a finite enumeration can determine what enumerate has discovered about the underlying iterable.
participants (5)
-
Andrew Barnert
-
Ilya Kamenshchikov
-
Serhiy Storchaka
-
Stephen J. Turnbull
-
Steven D'Aprano