Mailman 3 Re: Python-Dev Digest, Vol 201, Issue 1 - Python-Dev

2 Apr 2020

      One potentially serious question: what should `enumerate.__reversed__`
...
do when given a starting value?
    reversed(enumerate('abc', 1))
Should that yield...?
    # treat the start value as a start value
    (1, 'c'), (0, 'b'), (-1, 'a')
    # treat the start value as an end value
    (3, 'c'), (2, 'b'), (1, 'a')
Something else?
My preference would be to treat the starting value as an ending value.
My idea is largely that as long as any iterable has defined length and
order, it should be reversible, and this should yield the same as if I did
this reverse countdown with some pointer.
It just so happens that `enumerate` depends fully in this regard on the
iterable it enumerates, if it has defined length, then enumerate will also
have defined length. Maybe enumerate should also start defining `len`
whenever underlying iterable supports it.
Regarding the starting value, I think it is important to keep symmetry:
enumerate('abc', 1) is  [ (1, 'a'), (2, 'b'), (3, 'c') ], therefore
reversed(enumerate('abc', 1)) should be [(3, 'c'), (2, 'b'), (1, 'a')]
Best Regards,
--
Ilya Kamen
On Thu, Apr 2, 2020 at 5:55 AM python-dev-request@python.org wrote:
...
Send Python-Dev mailing list submissions to
        python-dev@python.org
To subscribe or unsubscribe via the World Wide Web, visit
        https://mail.python.org/mailman3/lists/python-dev.python.org/
or, via email, send a message with subject or body 'help' to
        python-dev-request@python.org
You can reach the person managing the list at
        python-dev-owner@python.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Python-Dev digest..."
Today's Topics:

Re: reversed enumerate (Steven D'Aprano)
Re: [Python-ideas] Re: reversed enumerate (Andrew Barnert)
Re: [Python-ideas] Re: reversed enumerate (Andrew Barnert)

Date: Thu, 2 Apr 2020 13:20:02 +1100
From: Steven D'Aprano steve@pearwood.info
Subject: [Python-Dev] Re: reversed enumerate
To: python-dev@python.org
Cc: python-ideas@python.org
Message-ID: 20200402022000.GC16830@ando.pearwood.info
Content-Type: text/plain; charset=us-ascii
Hi Ilya,
I'm not sure that this mailing list (Python-Dev) is the right place for
this discussion, I think that Python-Ideas (CCed) is the correct place.
For the benefit of Python-Ideas, I have left your entire post below, to
establish context.
[Ilya]
...
I needed reversed(enumerate(x: list)) in my code, and have discovered
that it wound't work. This is disappointing because operation is well
defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream. Consider:
def values():
    while True:
        yield random.random()

a, b = reversed(enumerate(values())

What should the first pair of (a, b) be?
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
On the other hand, I think that there is a fairly lightweight work
around. Define a helper function:
def countdown(n):
    while True:
        yield n
        n -= 1

then call it like this:
# reversed(enumerate(seq))
zip(countdown(len(seq)-1), reversed(seq)))

So it isn't terribly hard to work around this. But I agree that it would
be nice if enumerate encapsulated this for the caller.
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))

Should that yield...?
# treat the start value as a start value
(1, 'c'), (0, 'b'), (-1, 'a')

# treat the start value as an end value
(3, 'c'), (2, 'b'), (1, 'a')

Something else?
My preference would be to treat the starting value as an ending value.
Steven
On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
...
Hi,
I needed reversed(enumerate(x: list)) in my code, and have discovered
that
...
it wound't work. This is disappointing because operation is well defined.
It is also well defined for str type, range, and - in principle, but not
yet in practice - on dictionary iterators - keys(), values(), items() as
dictionaries are ordered now.
It would also be well defined on any user type implementing __iter__,
__len__, __reversed__ - think numpy arrays, some pandas dataframes,
tensors.
...
That's plenty of usecases, therefore I guess it would be quite useful to
avoid hacky / inefficient solutions like described here:
https://code.activestate.com/lists/python-list/706205/.
If deemed useful, I would be interested in implementing this, maybe
together with __reversed__ on dict keys, values, items.
Best Regards,
Ilya Kamen

p.s.
*Sketch* of what I am proposing:
class reversible_enumerate:
def __init__(self, iterable):
    self.iterable = iterable
    self.ctr = 0

def __iter__(self):
    for e in self.iterable:
        yield self.ctr, e
        self.ctr += 1

def __reversed__(self):
    try:
        ri = reversed(self.iterable)
    except Exception as e:
        raise Exception(
            "enumerate can only be reversed if iterable to

enumerate can be reversed and has defined length."
            ) from e
    try:
        l = len(self.iterable)
    except Exception as e:
        raise Exception(
            "enumerate can only be reversed if iterable to

enumerate can be reversed and has defined length."
            ) from e
    indexes = range(l-1, -1, -1)
    for i, e in zip(indexes, ri):
        yield i, e

for i, c in reversed(reversible_enumerate("Hello World")):
    print(i, c)
for i, c in reversed(reversible_enumerate([11, 22, 33])):
print(i, c)

...

Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUDH...
...
Code of Conduct: http://python.org/psf/codeofconduct/

Date: Wed, 1 Apr 2020 20:42:12 -0700
From: Andrew Barnert abarnert@yahoo.com
Subject: [Python-Dev] Re: [Python-ideas] Re: reversed enumerate
To: Steven D'Aprano steve@pearwood.info
Cc: python-dev@python.org, python-ideas@python.org
Message-ID: 647BE271-D7ED-4E50-A807-3D748B00FF1D@yahoo.com
Content-Type: text/plain; charset=utf-8
Before jumping in:
In many cases, when you want to reverse an enumerate, it’s small and
fixed-sized, so there’s a trivial way to do this: Just store the enumerate
iterator in a tuple, and tuples are reversible.
for idx, value in reversed(tuple(enumerate(stuff))):
But of course there are some cases where this isn’t appropriate, like
enumerating a fixed-size but huge input.
...
On Apr 1, 2020, at 19:23, Steven D'Aprano steve@pearwood.info wrote:
[Ilya]
...
I needed reversed(enumerate(x: list)) in my code, and have discovered
that it wound't work. This is disappointing because operation is well
defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream.
...
...
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
Agreed—but this is just a small piece of a much wider issue. Today,
enumerate is always an Iterator. It’s never reversible. But it’s also not
sized, or subscriptable, or in-testable, even if you give it inputs that
are. And it’s not just enumerate—the same is true for map, filter, zip,
itertools.islice, itertools.dropwhile, etc.
There’s no reason these things couldn’t all be views, just like the
existing dict views (and other things like memoryview and third-party
things like numpy array slices). In fact, they already are in Swift, and
will be in C++20.
...
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
Actually, that doesn’t work—it has to be Sized as well.
More generally, it’s rarely _quite_ as simple as just “views support the
same operations as the things they view”. An enumerate can be a Sequence if
its input is, but a filter can’t. A map with multiple inputs isn’t
Reversible unless they’re all not just Reversible but Sized, although a map
with only one input doesn’t need it to be Sized. And so on. But none of
these things are hard, it’s just a bunch of work to go through all the
input types for all the view types and write up the rules. (Or steal them
from another language or library that already did that work…)
...
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
Agreed. I don’t think we need to wait until someone designs and writes a
complete viewtools library and submits it for stdlib inclusion before we
can consider adding just one extension to one iterator. But I do think we
want to add the one(s) that are most useful if any, not just whichever ones
people think of first. I’ve personally wanted to reverse a map or a filter
more often than an enumerate, but examples would easily convince me that
that’s just me, and reversing enumerate is more needed.
...
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))
I don’t think this is a problem. When you reversed(tuple(enumerate('abc',
1))) today, what do you get? You presumably don’t even need to look that up
or try it out. It would be pretty confusing if it were different without
the tuple.

Subject: Digest Footer

Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

End of Python-Dev Digest, Vol 201, Issue 1

Re: Python-Dev Digest, Vol 201, Issue 1

Ilya Kamenshchikov

Best Regards,

tags (0)

participants (1)