[Python-checkins] r51297 - peps/trunk/pep-0357.txt

[neal.norwitz]

Author: neal.norwitz
Date: Tue Aug 15 08:27:32 2006
New Revision: 51297

Modified:
   peps/trunk/pep-0357.txt
Log:
Updates from Travis.  I modified his version based on a followup mail that
stated: As long as the object returned is a proper sub-class then it should
work fine.  The Exact checks should be removed from both the code and the PEP.


Modified: peps/trunk/pep-0357.txt
==============================================================================
--- peps/trunk/pep-0357.txt	(original)
+++ peps/trunk/pep-0357.txt	Tue Aug 15 08:27:32 2006
@@ -45,31 +45,68 @@
 Proposal
  
     Add an nb_index slot to PyNumberMethods, and a corresponding
-    __index__ special method.  Objects could define a function to place
-    in the nb_index slot that returns an appropriate C-integer (Py_ssize_t
-    after PEP 353).  This C-integer will be used whenever Python needs
-    one such as in PySequence_GetSlice, PySequence_SetSlice, and
-    PySequence_DelSlice.  
+    __index__ special method.  Objects could define a function to
+    place in the nb_index slot that returns a Python integer
+    (either an int or a long). This integer can 
+    then be appropriately converted to a Py_ssize_t value whenever 
+    Python needs one such as in PySequence_GetSlice, 
+    PySequence_SetSlice, and PySequence_DelSlice.
 
 Specification:
     
-    1) The nb_index slot will have the signature
+    1) The nb_index slot will have the following signature  
 
-       Py_ssize_t index_func (PyObject *self)
+       PyObject *index_func (PyObject *self)
+
+       The returned object must be a Python IntType or 
+       Python LongType. NULL should be returned on
+       error with an appropriate error set. 
 
     2) The __index__ special method will have the signature
 
        def __index__(self):
            return obj
        
-       where obj must be either an int or a long. 
+       where obj must be either an int or a long.
+
+    3) 3 new abstract C-API functions will be added 
 
-    3) A new C-API function PyNumber_Index will be added with signature
+       a) The first checks to see if the object supports the index
+          slot and if it is filled in. 
 
-       Py_ssize_t PyNumber_Index (PyObject *obj)
+          int PyIndex_Check(obj)
 
-       which will return obj->ob_type->tp_as_number->nb_index(obj) if it is available. 
-       A -1 will be returned and an exception set on an error. 
+          This will return true if the object defines the nb_index
+          slot.  
+
+       b) The second is a simple wrapper around the nb_index call that
+          raises PyExc_TypeError if the call is not available or if it
+          doesn't return an int or long.  Because the
+          PyIndex_Check is performed inside the PyNumber_Index call
+          you can call it directly and manage any error rather than
+          check for compatibility first.
+
+          PyObject *PyNumber_Index (PyObject *obj)
+
+       c) The third call helps deal with the common situation of
+          actually needing a Py_ssize_t value from the object to use for
+          indexing or other needs.
+
+          Py_ssize_t PyNumber_AsSsize_t(PyObject *obj, PyObject *exc)
+
+          The function calls the nb_index slot of obj if it is
+          available and then converts the returned Python integer into
+          a Py_ssize_t value.  If this goes well, then the value is
+          returned.  The second argument allows control over what
+          happens if the integer returned from nb_index cannot fit
+          into a Py_ssize_t value.
+
+          If exc is NULL, then the returnd value will be clipped to
+          PY_SSIZE_T_MAX or PY_SSIZE_T_MIN depending on whether the
+          nb_index slot of obj returned a positive or negative
+          integer.  If exc is non-NULL, then it is the error object
+          that will be set to replace the PyExc_OverflowError that was
+          raised when the Python integer or long was converted to Py_ssize_t.
 
     4) A new operator.index(obj) function will be added that calls
        equivalent of obj.__index__() and raises an error if obj does not implement
@@ -90,13 +127,19 @@
        slots for subscript access and use a special-check for integers to 
        check for the slot as well.
 
-    5) Add the nb_index slot to integers and long_integers.
+    5) Add the nb_index slot to integers and long_integers 
+       (which just return themselves)
 
     6) Add PyNumber_Index C-API to return an integer from any 
        Python Object that has the nb_index slot.  
 
     7) Add the operator.index(x) function.
 
+    8) Alter arrayobject.c and mmapmodule.c to use the new C-API for their
+       sub-scripting and other needs. 
+
+    9) Add unit-tests
+
 
 Discussion Questions
 
@@ -129,13 +172,22 @@
     for examples of names that were suggested such as "__discrete__" and 
     "__ordinal__".
 
-    Why return Py_ssize_t from nb_index?  
+    Why return PyObject * from nb_index?  
 
-    The nb_index slot is primarily intended to return an integer
-    needed by the sequence interface.  In Python 2.5 this is
-    Py_ssize_t.  As this is the primary purpose of the slot, it
-    makes sense to return the C-integer directly and not wrapped
-    in a Python int object. 
+    Intially Py_ssize_t was selected as the return type for the
+    nb_index slot.  However, this led to an inability to track and
+    distinguish overflow and underflow errors without ugly and brittle
+    hacks. As the nb_index slot is used in at least 3 different ways 
+    in the Python core (to get an integer, to get a slice end-point, 
+    and to get a sequence index), there is quite a bit of flexibility
+    needed to handle all these cases.  The importance of having the
+    necessary flexibility to handle all the use cases is critical. 
+    For example, the initial implementation that returned Py_ssize_t for
+    nb_index led to the discovery that on a 32-bit machine with >=2GB of RAM 
+    s = 'x' * (2**100) works but len(s) was clipped at 2147483647.
+    Several fixes were suggested but eventually it was decided that
+    nb_index needed to return a Python Object similar to the nb_int
+    and nb_long slots in order to handle overflow correctly.
 
     Why can't __index__ return any object with the nb_index method?