Coercion and comparison of numbers
I just received a bug report for mx.Number which revealed a probelm with the comparison code in Python 2.1. Looking at the code it seems that one of my original coercion patches did not make it into the core. I added a new API PyNumber_Compare() knows about the new coercion mechanism and should be called for numbers instead of trying coercion in PyObject_Compare(). Was this part of the coercion patch left out on purpose or a simple oversight ? I hope the latter... -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/
Hard to say. I don't think I paid very close attention to your patch; Neil did, but I changed a lot of the code around coercions and comparisons in order to implement rich comparisons. So, several things may have happened: Neil lost it; Neil decided against it; or I ripped it out. Can you elucidate me regarding the issues? (If there's code, please quote it or link to a specific patch.) Since the concept of "number" is ill-defined at best, when exactly should PyNumber_Compare() be called? What is it supposed to do? Does it need a rich cousin? --Guido van Rossum (home page: http://www.python.org/~guido/)
[MAL]
I remember the API. I don't remember what happened to it. Guido might have dropped it or I might have taken it out thinking the comparison issues would be sorted out by Guido. Why is a new API needed? Why can't PyObject_Compare() do the right thing (ie. not coerce new style numbers)? Neil
Neil Schemenauer wrote:
Good; so there's a chance for getting it back in :-)
Why is a new API needed? Why can't PyObject_Compare() do the right thing (ie. not coerce new style numbers)?
I think the reason for implementing number compares as separate API was to simply shift out code from PyObject_Compare() into a new function, not so much motivated by some higher level need to do number compares. [Guido]
The reasoning is simple: the coercion patches basically pass control over coercion down to the APIs in question and thus provide the type with more information to choose from. This is currently implemented in 2.1 for all number methods, but not for number comparisons which do have the same problems with centralized coercion as e.g. __add__ or other binary operators. Here's part of the original patch: --- Include/orig/abstract.h Wed May 13 00:28:58 1998 +++ Include/abstract.h Thu May 21 12:31:55 1998 @@ -447,11 +447,18 @@ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx This function always succeeds. */ - PyObject *PyNumber_Add Py_PROTO((PyObject *o1, PyObject *o2)); + PyObject *PyNumber_Compare Py_PROTO((PyObject *o1, PyObject *o2)); + + /* + Returns the result of comparing o1 and o2, or null on failure. + This is the equivalent of the Python expression: cmp(o1,o2). + */ + + PyObject *PyNumber_Add Py_PROTO((PyObject *o1, PyObject *o2)); /* Returns the result of adding o1 and o2, or null on failure. This is the equivalent of the Python expression: o1+o2. [...] } +/* Emulate old method for comparing numeric types using coercion and + tp_compare. If coercion doesn't work, we use the type names as + comparison basis (like PyObject_Compare() does too). */ + +static PyObject * +_PyNumber_OldstyleCompare(PyObject *v, + PyObject *w) +{ + int err; + + DPRINTF("_PyNumber_OldstyleCompare(%s at 0x%lx, %s at 0x%lx);\n", + v->ob_type->tp_name,(long)v, + w->ob_type->tp_name,(long)w); + err = PyNumber_CoerceEx(&v, &w); + if (err < 0) + return NULL; + else if (err == 0 && v->ob_type->tp_compare) { + int cmp; + + cmp = (*v->ob_type->tp_compare)(v, w); + /* XXX Test for errors ? Looks like C types cannot raise + exceptions in the compare slot... */ + Py_DECREF(v); + Py_DECREF(w); + DPRINTF(" compare slot returned: %i",cmp); + return PyInt_FromLong(cmp); + } + DPRINTF(" using type names for comparison\n"); + return PyInt_FromLong(strcmp(v->ob_type->tp_name, + w->ob_type->tp_name)); +} + +PyObject * +PyNumber_Compare(v, w) + PyObject *v, *w; +{ + DPRINTF("PyNumber_Compare(%s at 0x%lx, %s at 0x%lx);\n", + v->ob_type->tp_name,(long)v, + w->ob_type->tp_name,(long)w); + BINOP("__cmp__", "__rcmp__", PyNumber_Compare); + return _PyNumber_BinaryOperation(v,w, + NB_SLOT(nb_cmp), + "cmp()"); +} + [...] +static PyObject * +_PyNumber_BinaryOperation(PyObject *v, + PyObject *w, + const int op_slot, + const char *operation) +{ + PyNumberMethods *mv, *mw; + register PyObject *x; + register binaryfunc *slot; + int c; ... + /* When using old coercion, make sure that the requested slot + is available on old style numbers or use an emulation. */ + if (op_slot > NB_SLOT(nb_hex)) { + + /* Emulation hooks: */ + if (op_slot == NB_SLOT(nb_cmp)) + return _PyNumber_OldstyleCompare(v,w); + + goto badOperands; + } [...] int PyObject_Compare(v, w) PyObject *v, *w; { PyTypeObject *tp; @@ -291,27 +294,30 @@ PyObject_Compare(v, w) Py_DECREF(res); PyErr_SetString(PyExc_TypeError, "comparison did not return an int"); return -1; } - c = PyInt_AsLong(res); + c = PyInt_AS_LONG(res); Py_DECREF(res); return (c < 0) ? -1 : (c > 0) ? 1 : 0; } if ((tp = v->ob_type) != w->ob_type) { - if (tp->tp_as_number != NULL && - w->ob_type->tp_as_number != NULL) { - int err; - err = PyNumber_CoerceEx(&v, &w); - if (err < 0) + if (tp->tp_as_number != NULL || + w->ob_type->tp_as_number != NULL) { + PyObject *res; + int c; + res = PyNumber_Compare(v,w); + if (res == NULL) return -1; - else if (err == 0) { - int cmp = (*v->ob_type->tp_compare)(v, w); - Py_DECREF(v); - Py_DECREF(w); - return cmp; + if (!PyInt_Check(res)) { + PyErr_SetString(PyExc_TypeError, + "comparison did not return an int"); + return -1; } + c = PyInt_AS_LONG(res); + Py_DECREF(res); + return (c < 0) ? -1 : (c > 0) ? 1 : 0; } return strcmp(tp->tp_name, w->ob_type->tp_name); } if (tp->tp_compare == NULL) return (v < w) ? -1 : 1; -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/
Hard to say. I don't think I paid very close attention to your patch; Neil did, but I changed a lot of the code around coercions and comparisons in order to implement rich comparisons. So, several things may have happened: Neil lost it; Neil decided against it; or I ripped it out. Can you elucidate me regarding the issues? (If there's code, please quote it or link to a specific patch.) Since the concept of "number" is ill-defined at best, when exactly should PyNumber_Compare() be called? What is it supposed to do? Does it need a rich cousin? --Guido van Rossum (home page: http://www.python.org/~guido/)
[MAL]
I remember the API. I don't remember what happened to it. Guido might have dropped it or I might have taken it out thinking the comparison issues would be sorted out by Guido. Why is a new API needed? Why can't PyObject_Compare() do the right thing (ie. not coerce new style numbers)? Neil
Neil Schemenauer wrote:
Good; so there's a chance for getting it back in :-)
Why is a new API needed? Why can't PyObject_Compare() do the right thing (ie. not coerce new style numbers)?
I think the reason for implementing number compares as separate API was to simply shift out code from PyObject_Compare() into a new function, not so much motivated by some higher level need to do number compares. [Guido]
The reasoning is simple: the coercion patches basically pass control over coercion down to the APIs in question and thus provide the type with more information to choose from. This is currently implemented in 2.1 for all number methods, but not for number comparisons which do have the same problems with centralized coercion as e.g. __add__ or other binary operators. Here's part of the original patch: --- Include/orig/abstract.h Wed May 13 00:28:58 1998 +++ Include/abstract.h Thu May 21 12:31:55 1998 @@ -447,11 +447,18 @@ xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx This function always succeeds. */ - PyObject *PyNumber_Add Py_PROTO((PyObject *o1, PyObject *o2)); + PyObject *PyNumber_Compare Py_PROTO((PyObject *o1, PyObject *o2)); + + /* + Returns the result of comparing o1 and o2, or null on failure. + This is the equivalent of the Python expression: cmp(o1,o2). + */ + + PyObject *PyNumber_Add Py_PROTO((PyObject *o1, PyObject *o2)); /* Returns the result of adding o1 and o2, or null on failure. This is the equivalent of the Python expression: o1+o2. [...] } +/* Emulate old method for comparing numeric types using coercion and + tp_compare. If coercion doesn't work, we use the type names as + comparison basis (like PyObject_Compare() does too). */ + +static PyObject * +_PyNumber_OldstyleCompare(PyObject *v, + PyObject *w) +{ + int err; + + DPRINTF("_PyNumber_OldstyleCompare(%s at 0x%lx, %s at 0x%lx);\n", + v->ob_type->tp_name,(long)v, + w->ob_type->tp_name,(long)w); + err = PyNumber_CoerceEx(&v, &w); + if (err < 0) + return NULL; + else if (err == 0 && v->ob_type->tp_compare) { + int cmp; + + cmp = (*v->ob_type->tp_compare)(v, w); + /* XXX Test for errors ? Looks like C types cannot raise + exceptions in the compare slot... */ + Py_DECREF(v); + Py_DECREF(w); + DPRINTF(" compare slot returned: %i",cmp); + return PyInt_FromLong(cmp); + } + DPRINTF(" using type names for comparison\n"); + return PyInt_FromLong(strcmp(v->ob_type->tp_name, + w->ob_type->tp_name)); +} + +PyObject * +PyNumber_Compare(v, w) + PyObject *v, *w; +{ + DPRINTF("PyNumber_Compare(%s at 0x%lx, %s at 0x%lx);\n", + v->ob_type->tp_name,(long)v, + w->ob_type->tp_name,(long)w); + BINOP("__cmp__", "__rcmp__", PyNumber_Compare); + return _PyNumber_BinaryOperation(v,w, + NB_SLOT(nb_cmp), + "cmp()"); +} + [...] +static PyObject * +_PyNumber_BinaryOperation(PyObject *v, + PyObject *w, + const int op_slot, + const char *operation) +{ + PyNumberMethods *mv, *mw; + register PyObject *x; + register binaryfunc *slot; + int c; ... + /* When using old coercion, make sure that the requested slot + is available on old style numbers or use an emulation. */ + if (op_slot > NB_SLOT(nb_hex)) { + + /* Emulation hooks: */ + if (op_slot == NB_SLOT(nb_cmp)) + return _PyNumber_OldstyleCompare(v,w); + + goto badOperands; + } [...] int PyObject_Compare(v, w) PyObject *v, *w; { PyTypeObject *tp; @@ -291,27 +294,30 @@ PyObject_Compare(v, w) Py_DECREF(res); PyErr_SetString(PyExc_TypeError, "comparison did not return an int"); return -1; } - c = PyInt_AsLong(res); + c = PyInt_AS_LONG(res); Py_DECREF(res); return (c < 0) ? -1 : (c > 0) ? 1 : 0; } if ((tp = v->ob_type) != w->ob_type) { - if (tp->tp_as_number != NULL && - w->ob_type->tp_as_number != NULL) { - int err; - err = PyNumber_CoerceEx(&v, &w); - if (err < 0) + if (tp->tp_as_number != NULL || + w->ob_type->tp_as_number != NULL) { + PyObject *res; + int c; + res = PyNumber_Compare(v,w); + if (res == NULL) return -1; - else if (err == 0) { - int cmp = (*v->ob_type->tp_compare)(v, w); - Py_DECREF(v); - Py_DECREF(w); - return cmp; + if (!PyInt_Check(res)) { + PyErr_SetString(PyExc_TypeError, + "comparison did not return an int"); + return -1; } + c = PyInt_AS_LONG(res); + Py_DECREF(res); + return (c < 0) ? -1 : (c > 0) ? 1 : 0; } return strcmp(tp->tp_name, w->ob_type->tp_name); } if (tp->tp_compare == NULL) return (v < w) ? -1 : 1; -- Marc-Andre Lemburg ______________________________________________________________________ Company & Consulting: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/
participants (3)
-
Guido van Rossum -
M.-A. Lemburg -
Neil Schemenauer