[Python-Dev] PEP for adding an sq_index slot so that any object, a or b, can be used in X[a:b] notation

[Guido van Rossum]

On 2/9/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> Guido seemed accepting to this idea about 9 months ago when I spoke to
> him.  I finally got around to writing up the PEP.   I'd really like to
> get this into Python 2.5 if possible.

Excellent! I was just going over the 2.5 schedule with Neal Norwitz
last night, and looking back in my slides for OSCON 2005 I noticed
this idea, and was wondering if you still wanted it. I'm glad the
answer is yes!

BTW do you also still want to turn ZeroDivisionError into a warning
(that is changed into an error by default)? That idea shared a slide
and I believe it was discussed in the same meeting you & I and some
others had in San Mateo last summer.

I'll comment on the PEP in-line. I've assigned it number 357 and checked it in.

<meta-remark>

In the past, the protocol for aqcuiring a PEP number has been to ask
the PEP coordinators (Barry Warsaw and David Goodger) to assign one. I
believe that we could simplify this protocol to avoid necessary
involvement of the PEP coordinators; all that is needed is someone
with checkin privileges. I propose the following protocol:

1. In the peps directory, do a svn sync.

2. Look at the files that are there and the contents of pep-0000.txt.
This should provide you with the last PEP number in sequence, ignoring
the out-of-sequence PEPs (666, 754, and 3000). The reason to look in
PEP 0 is that it is conceivable that a PEP number has been reserved in
the index but not yet committed, so you should use the largest number.

3. Add 1 to the last PEP number. This gives your new PEP number, NNNN.

4. Using svn add and svn commit, check in the file pep-NNNN.txt (use
%04d to format the number); the contents can be a minimal summary or
even just headers. If this succeeds, you have successfully assigned
yourself PEP number NNNN. Exit.

5. If you get an error from svn about the commit, someone else was
carrying out the same protocol at the same time, and they won the
race. Start over from step 1.

I suspect the PEP coordinators have informally been using this
protocol amongst themseles -- and amongst the occasional developer who
bypassed the "official" protocol, like I've done in the past and like
Neal Norwitz did last night with the Python 2.5 release schedule, PEP
356. I'm simply extending the protocol to all developers with checkin
permissions. For PEP authors without checkin permissions, nothing
changes, except that optionally if they don't get a timely response
from the PEP coordinators, they can ask someone else with checkin
permissions.

</meta-remark>


> PEP:  ###
> Title:  Allowing any object to be used for slicing
> Version:  $Revision 1.1$
> Last Modified: $Date: 2006/02/09 $
> Author: Travis Oliphant <oliphant at ee.byu.edu>
> Status: Draft
> Type:  Standards Track
> Created:  09-Feb-2006
> Python-Version:  2.5
>
> Abstract
>
>    This PEP proposes adding an sq_index slot in PySequenceMethods and
>    an __index__ special method so that arbitrary objects can be used
>    in slice syntax.
>
> Rationale
>
>    Currently integers and long integers play a special role in slice
>    notation in that they are the only objects allowed in slice
>    syntax. In other words, if X is an object implementing the sequence
>    protocol, then X[obj1:obj2] is only valid if obj1 and obj2 are both
>    integers or long integers.  There is no way for obj1 and obj2 to
>    tell Python that they could be reasonably used as indexes into a
>    sequence.  This is an unnecessary limitation.
>
>    In NumPy, for example, there are 8 different integer scalars
>    corresponding to unsigned and signed integers of 8, 16, 32, and 64
>    bits.  These type-objects could reasonably be used as indexes into
>    a sequence if there were some way for their typeobjects to tell
>    Python what integer value to use.
>
> Proposal
>
>    Add a sq_index slot to PySequenceMethods, and a corresponding
>    __index__ special method.  Objects could define a function to
>    place in the sq_index slot that returns an C-integer for use in
>    PySequence_GetSlice, PySequence_SetSlice, and PySequence_DelSlice.

Shouldn't this slot be in the PyNumberMethods extension? It feels more
like a property of numbers than of a property of sequences. Also, the
slot name should then probably be nb_index.

There's also an ambiguity when using simple indexing. When writing
x[i] where x is a sequence and i an object that isn't int or long but
implements __index__, I think i.__index__() should be used rather than
bailing out. I suspect that you didn't think of this because you've
already special-cased this in your code -- when a non-integer is
passed, the mapping API is used (mp_subscript). This is done to
suppose extended slicing. The built-in sequences (list, str, unicode,
tuple for sure, probably more) that implement mp_subscript should
probe for nb_index before giving up. The generic code in
PyObject_GetItem should also check for nb_index before giving up.

> Implementation Plan
>
>    1) Add the slots
>
>    2) Change the ISINT macro in ceval.c to accomodate objects with the
>    index slot defined.
>
>    3) Change the _PyEval_SliceIndex function to accomodate objects
>    with the index slot defined.

I think all sequence objects that implement mp_subscript should
probably be modified according to the lines I sketched above.

> Possible Concerns
>
>    Speed:
>
>    Implementation should not slow down Python because integers and long
>    integers used as indexes will complete in the same number of
>    instructions.  The only change will be that what used to generate
>    an error will now be acceptable.
>
>    Why not use nb_int which is already there?
>
>    The nb_int, nb_oct, and nb_hex methods are used for coercion.
>    Floats have these methods defined and floats should not be used in
>    slice notation.
>
> Reference Implementation
>
>    Available on PEP acceptance.

This is very close to acceptance. I think I'd like to see the patch
developed and submitted to SF (and assigned to me) prior to
acceptance.

> Copyright
>
>    This document is placed in the public domain

If you agree with the above comments, please send me an updated
version of the PEP and I'll check it in over the old one, and approve
it. Then just use SF to submit the patch etc.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)