[Python-ideas] Where did we go wrong with negative stride?

Tue Oct 29 00:13:18 CET 2013

On 29 Oct 2013 08:41, "Guido van Rossum" <guido at python.org> wrote:
>
> I'm not sure I like new syntax. We'd still have to find a way to
represent this with slice() and also with range().

Those are much easier: we can just add a "reverse=False" keyword-only
argument.

However, I realised that given the need to appropriately document these
function signatures and the precedent set by sorted (where the reverse flag
is essentially an optimisation trick that avoids a separate reversal
operation) the cleaner interpretation of such an argument is for:

    range(i, j, k, reverse=True)

to effectively mean:

    range(i, j, k)[::-1]

and for:

    s[slice(i, j, k, reverse=True)]

to effectively mean:

    s[i:j:k][::-1]

range and slice would handle the appropriate start/stop/step calculations
under the hood and hence be backwards compatible with existing container
implementations and other code.

This approach also means we could avoid addressing the slice reversal
syntax question for 3.4, and revisit it in the 3.5 time frame (and ditto
for deprecating negative strides). However, the idea of just allowing
keyword args to be passed to the slice builtin in the slice syntax did
occur to me:

    s[i:j:k:reverse=True]

Cheers,
Nick.

>
>
> On Mon, Oct 28, 2013 at 3:29 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>
>>
>> On 29 Oct 2013 06:21, "Oscar Benjamin" <oscar.j.benjamin at gmail.com>
wrote:
>> >
>> > On 28 October 2013 15:04, Brett Cannon <brett at python.org> wrote:
>> > >
>> > > On Mon, Oct 28, 2013 at 10:49 AM, Oscar Benjamin
>> > > <oscar.j.benjamin at gmail.com> wrote:
>> > >>
>> > >> On 28 October 2013 13:57, Brett Cannon <brett at python.org> wrote:
>> > >>
>> > >> I think that the numpy mailing lists should be consulted before any
>> > >> decisions are made. As Antoine says: if you've never noticed this
>> > >> behaviour before then it obviously doesn't matter to you that much
so
>> > >> why the rush to deprecate it?
>> > >
>> > > I'm not saying not to talk to them, but I also don't think we should
>> > > necessarily not change it because no one uses it either. If it's
wide spread
>> > > then sure, we just live with it. It's always a balancing act of
fixing for
>> > > future code vs. pain of current code. I'm just saying we shouldn't
dismiss
>> > > changing this out of hand because you are so far the only person who
has
>> > > relied on this.
>> >
>> > Anyone who has used negative strides and non-default start/stop is
>> > relying on it. It has been a core language feature since long before I
>> > started using Python.
>> >
>> > Also I'm not the only person to point out that a more common problem
>> > is with wraparound when doing something like a[:n]. That is a more
>> > significant problem with slicing and I don't understand why the
>> > emphasis here is all on the negative strides rather then negative
>> > indexing. Using negative indices to mean "from the end" is a mistake
>> > and it leads to things like this:
>> >
>> > >>> a = 'abcdefg'
>> > >>> for n in reversed(range(7)):
>> > ...     print(n, a[:-n])
>> > ...
>> > 6 a
>> > 5 ab
>> > 4 abc
>> > 3 abcd
>> > 2 abcde
>> > 1 abcdef
>> > 0
>> >
>> > You can do something like a[:-n or None] to correctly handle zero but
>> > that still does the wrong thing when n is negative.
>> >
>> > Also why do you get an error when your index goes off one end of the
>> > array but not when it goes off the other?
>> >
>> > >>> a[len(a)]
>> > Traceback (most recent call last):
>> >   File "<stdin>", line 1, in <module>
>> > IndexError: string index out of range
>> > >>> a[-1]
>> > 'g'
>> >
>> > I have never been in a situation where I was writing code and didn't
>> > know whether I wanted to slice/index from the end or the beginning at
>> > coding time. I would much rather that a[-1] be an error and have an
>> > explicit syntax for indexing from the end.
>> >
>> > I have never found this wraparound to be useful and I think that if
>> > there is a proposal to change slicing in a backward incompatible way
>> > then it should be to something that is significantly better by solving
>> > these real problems. I have often had bugs or been forced to write
>> > awkward code because of these. The negative slicing indices may be a
>> > bit harder to reason about but it has never actually caused me any
>> > problems.
>> >
>> > Something like the matlab/pike syntax would fix these problems as well
>> > as making it possible to use the same indices for negative stride
>> > slices. That would be worth a deprecation process in my opinion. The
>> > specific suggestion so far does not have enough of an advantage to
>> > justify breaking anyone's code IMO.
>> >
>> > > As for the rush, it's because 3.4b1 is approaching and if this slips
to 3.5
>> > > that's 1.5 years of deprecation time lost for something that doesn't
have a
>> > > syntactic break to help you discover the change in semantics.
>> > >>
>> > >> > +1 on doing a deprecation in 3.4.
>> > >>
>> > >> -1 on any deprecation without a clear plan for a better syntax.
Simply
>> > >> changing the semantics of the current syntax would bring in who
knows
>> > >> how many off-by-one errors for virtually no benefit.
>> > >
>> > > The deprecation would be in there from now until Python 4 so it
wouldn't be
>> > > sudden (remember that we are on a roughly 18 month release cycle, so
if this
>> > > went into 3.4 that's 7.5 years until this changes in Python 4). And
there's
>> > > already a future-compatible way to change your code to get the same
results
>> > > in the end that just require more explicit steps/code.
>> >
>> > This argument would be more persuasive if you said: "the new better
>> > syntax that solves many of the slicing problems will be introduced
>> > *now*, and the old style/syntax will be deprecated later".
>>
>> Indeed. I like Terry's proposed semantics, so if we can give that a new
syntax, we can just give range and slice appropriate "reverse=False"
keyword arguments like sorted and list.sort, and never have to deprecate
negative strides (although negative strides would be disallowed when
reverse=True).
>>
>> For example:
>>
>> s[i:j:k] - normal forward slice
>> s[i:j:<<k] - reversed slice (with i and j as left/right rather than
start/stop)
>>
>> Reversing with unit stride could be:
>> s[i:j:<<]
>>
>> When reverse=True, start, stop and step for the range or slice would be
calculated as follows:
>>
>> start = len(s)-1 if j is None else j-1
>> stop = -1 if i is None else i-1
>> step = -k
>>
>> It doesn't solve the -j vs len(s)-j problem for the end index, but I
think it's still more intuitive for the reasons Tim and Terry gave.
>>
>> Cheers,
>> Nick.
>>
>> >
>> >
>> > Oscar
>> > _______________________________________________
>> > Python-ideas mailing list
>> > Python-ideas at python.org
>> > https://mail.python.org/mailman/listinfo/python-ideas
>>
>>
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131029/fd804850/attachment.html>