[Python-ideas] Possible new slice behaviour? Was ( Negative slice discussion.)

Tue Nov 5 12:02:02 CET 2013

On 5 Nov 2013 11:48, "Ron Adam" <ron3200 at gmail.com> wrote:
>
> On 11/04/2013 04:40 PM, Nick Coghlan wrote:
>>
>>
>> On 5 Nov 2013 03:35, "Ron Adam" <ron3200 at gmail.com> wrote:
>> >
>> >
>> > This is one solution to what we can do to make slices both easier to
understand and work in a much more consistent and flexible way.
>> >
>> > This matches the slice semantics that Guido likes.
>> > (possibly for python 4.)
>> >
>> > The ability to pass callables along with the slice creates an easy and
clean way to add new indexing modes.  (As Nick suggested.)
>>
>> Tuples can't really be used for this purpose, since that's incompatible
with multi-dimensional indexing.
>
> Are there plans for pythons builtin types to use multidimensional
indexing?

Yes, memoryview already has some support for multidimensional array shapes,
and that's likely to be enhanced further in 3.5.

> I don't think what I'm suggesting would create an issue with it in
either.  It may even be complementary.
>
> Either I'm missing something, or you aren't quite understanding where the
changes I'm suggesting are to be made.   As long as the change is made
local to the object that uses it, it won't effect any other types uses of
slices.    And what is passed in a tuple is different from specifying the
meaning of a tuple.

You're proposing a mechanism for slice index customisation that would be
ambiguous and thoroughly confusing when used to define a slice as part of a
multidimensional array access.

Remember, the Ellipsis was first added specifically as part of
multi-dimensional indexing notation for the scientific community. Even
though the stdlib only partially supports it today, the conventions for
multidimensional slicing are defined by NumPy and need to be taken into
account in future design changes.

> There may be other reasons this may not be a bad idea, but I can't think
of any myself at the moment.   Possibly because a callable passed with a
slice may alter the object, but that could be limited by giving the
callable a length instead of of the object itself.  But personally I like
that it's open ended and not limited by the syntax.

I like the idea of a passing a callable. I just think it should be an extra
optional argument to slice that is used to customise the result of calling
indices() rather than changing the type seen by the underlying container.

>
>
> Consider this...
>
> >>> a = list(range(10))
> >>> a
> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
> >>> b = list([a] * 3)
> >>> b
> [[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1,
2, 3, 4, 5, 6, 7, 8, 9]]
> >>> a[2:5, 1:2]
>
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: list indices must be integers, not tuple
>
> Python lists currently don't know what to do with a tuple.   In order to
do anything else, the __getitem__ and __setitem__ methods need to be
overridden.   For that reason, it can't cause an issue with anything as
long as the change is kept *local to the object(s)* that use it.

Except for all the humans that will have to read it, and the confusion of
applying it to multidimensional array operations.

>
> Making changes at the syntax level, or even slice level could be
disruptive though.   (This doesn't do that.)

And hence only works with types that have been updated to support it. We
already did that once for extended slicing support, so let's not do it
again when there are other alternatives available.

However, using a custom container type is a good way to experiment, so I've
gone back to not wanting to permit slice subclasses at this point (since
containment is sufficient when experimenting with a custom container).

>
> >>> class Foo:
> ...     def __getitem__(self, args):
> ...         print(args)
> ...
> >>> foo = Foo()
> >>> foo[1,2,3,4]
> (1, 2, 3, 4)
> >>> foo[1:2:3, 4:5:6, 7, 8, 9]
> (slice(1, 2, 3), slice(4, 5, 6), 7, 8, 9)
>
> The slice syntax already constructs a tuple if it gets a complex set of
argument.   That isn't being changed.
>
> The only thing that it does is expand what builtin types can accept
through the existing syntax.  It does not restrict, or make any change, at
a level that will prevent anything else from using that same syntax in
other ways.

Yes, I realise it requires changes to all the container types. That's one
of the problems with the idea.

So, yes, I did understand your proposal, and definitely like the general
idea of passing in a callable to customise the index calculations. I just
don't like the specifics of it, both because of the visual confusion with
multidimensional indexing and because we've been through this before with
extended slicing and requiring changes to every single container type is a
truly painful way to make a transition. Implementing a change through the
slice object instead would mean that any transition problems would be
isolated to using the new feature with containers that didn't call
slice.indices or the C API equivalent when calculating slice indices.

Cheers,
Nick.

>
>
> As a way to allow new-slices and the current slices together/overlap in a
transition period, we could just require one extra value to be passed,
which would cause a tuple to be created and the __getitem__ method could
then use the newer indexing on the slice.
>
>     s[i:j:k]               # current indexing.
>     s[i:j:k, '']            # new indexing...  Null string or None causes
tuple to be created.  (or a callable that takes a slice.)
>
>
>
>> However, I also agree containment would be a better way to go than
subclassing.
>>
>> I'm currently thinking that a fourth "adjust" argument to the slice
constructor may work, and call that from the indices method as:
>>
>>   def adjust_indices(start, stop, step, length):
>>    ...
>
> Currently the length adjustment is made by the __getitem__ method calling
the indices method as in this example.
>
> >>> class Foo(list):
> ...     def __getitem__(self, slc):
> ...         print(slc.indices(len(self)))
> ...
> >>> foo = Foo(range(10))
> >>> foo
> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
> >>> foo[:]
> (0, 10, 1)
> >>> foo[::-1]
> (9, -1, -1)                # The altered indices we don't like from the
indices method.
>
>
> So you don't need to add the fourth length argument if the change is made
in __getitem__ and __setitem__.
> Or possibly you can do it just in the slices, indices method.
>
>
>
>> The values passed in would be those from the slice constructor plus the
length passed to the indices method. The only preconditioning would be the
check for a non-zero step.
>
>
>> The result would be used as the result of the indices method.
>
>
> Did you see this part of the tests?
>
>
> >     # And various other combinations.
> >
> >     >>> s = Str('123456789')
> >
> >     >>> s[3, onei]                      # ones indexing
> >     '3'
> >
> >     >>> s[4:8, onei]                   # ones indexing with slice
> >     '4567'
> >
> >     >>> s[4:8, onei, openi]           # open interval
> >     '567'
> >
> >     >>> s[4:8, onei, closedi]         # closed interval
> >     '45678'
>
>
> These all were very easy to implement, and did not require any extra
logic added to the underlying __getitem__ code other than calling the
passed functions in the tuple.   It moves these cases out of the object
being sliced in a nice way.   Other ways of doing it would require keywords
and logic for each case to be included in the objects.
>
> Cheers,adjustment
>     Ron
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131105/769e6fc2/attachment.html>