[Matrix-SIG] Re: NumPy core dump
Charles G Waldman
cgw@alum.mit.edu
28 Jan 1999 23:19:34 -0500
The following message is a courtesy copy of an article
that has been posted to comp.lang.python as well.
Chad Scherrer <chscherr@indiana.edu> writes:
> import Numeric
> x=Numeric.array([2l,3l,4l]) # those are L's, not ones
> print x**2
> Python kicks me out with a segmentation fault and a core dump
This is due to the fact that PyNumber_Power defined in the Python core
(Objects/abstract.c) is a ternary function (the 3-argument version
pow(x,y,z) means x**y%z), whereas NumPy's {fast_,}umathmodule.c casts
it to a binary function and calls it with two arguments. The reason
different people have seen different behavior with Chad's example is
due to the fact that the third argument is basically random junk - in
whatever was lying around on the stack when PyNubmer_Power gets
called. The test at line 614 in Python/abstract.c (in the current CVS
version of 1.5.2) either fails and an exception is raised (if the third
argument "z" happens to point to a legal address in memory) or else if
"z" is wild enough the attempt to dereference it generates a
segmentation fault.
Either way, it's a misbehavior. Looking at the Numeric code, it is
clear that powers of general-object arrays are intended to be supported.
While investigating this problem I noticed that in fact the ternary
version of power is not implemented for arrays:
Python 1.5.2b2 (#17, Jan 13 1999, 00:17:47) [GCC egcs-2.91.60 19981201 (e on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import Numeric
>>> Numeric.__version__
'1.9'
>>> a = Numeric.array((2,3,4))
>>> b=Numeric.array((5,6,7))
>>> c=Numeric.array((4,5,6))
>>> a**b
array([ 32, 729, 16384])
>>> pow(a,b,c)
array([ 32, 729, 16384])
This should also get fixed.
The patch below fixes the core dump problem for powers of
general-object arrays, however it does not fix the fact that ternary
pow() does not work on arrays. The Right fix would take care of both
of these, hence the comment marked with XXX. When I have some more
time I'll try to code that up.
I found some other less-than-perfect behavior about the support of
long integers in NumPy; for instance:
Python 1.5.2b2 (#17, Jan 13 1999, 00:17:47) [GCC egcs-2.91.60 19981201 (e on linux2
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> from math import *
>>> sqrt(4L)
2.0
>>> from Numeric import *
>>> sqrt(4L)
Traceback (innermost last):
File "<stdin>", line 1, in ?
AttributeError: 'long int' object has no attribute 'sqrt'
There is certainly some more work to be done to make support of
general object arrays a little cleaner in NumPy.
===================================================================
RCS file: RCS/umathmodule.c,v
retrieving revision 1.1
diff --unified=2 -r1.1 umathmodule.c
--- umathmodule.c 1999/01/29 03:07:20 1.1
+++ umathmodule.c 1999/01/29 04:15:48
@@ -1710,4 +1710,13 @@
}
}
+/* XXX hack to work around the fact that PyNumber_Power in
+ * Python/abstract.c is ternary; but we call it as a binary function.
+ * This fixes the bug where array((1L,2L,3L))*2 sometimes dumps core
+ * However the ternary versions of PyNumber_Power should be properly
+ * supported, once this gets done we can eliminate this hack and also
+ * get the ternary version of pow() to work for arrays */
+static PyObject * PyNumber_PowerBinary(PyObject *v,PyObject *w){
+ return PyNumber_Power(v,w,Py_None);
+}
static PyUFuncGenericFunction add_functions[] = { UBYTE_add, SBYTE_add, SHORT_add, INT_add, LONG_add, FLOAT_add, DOUBLE_add, CFLOAT_add, CDOUBLE_add, NULL, };
static PyUFuncGenericFunction subtract_functions[] = { UBYTE_subtract, SBYTE_subtract, SHORT_subtract, INT_subtract, LONG_subtract, FLOAT_subtract, DOUBLE_subtract, CFLOAT_subtract, CDOUBLE_subtract, NULL, };
@@ -1748,5 +1757,5 @@
static void * conjugate_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)"conjugate", };
static void * remainder_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)fmod, (void *)fmod, (void *)PyNumber_Remainder, };
-static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_Power, };
+static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_PowerBinary, };
static void * absolute_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Absolute, };
static void * negative_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Negative, };
===================================================================
RCS file: RCS/fast_umathmodule.c,v
retrieving revision 1.1
diff --unified=2 -r1.1 fast_umathmodule.c
--- fast_umathmodule.c 1999/01/29 03:32:22 1.1
+++ fast_umathmodule.c 1999/01/29 04:15:49
@@ -1711,4 +1711,13 @@
}
}
+/* XXX hack to work around the fact that PyNumber_Power in
+ * Python/abstract.c is ternary; but we call it as a binary function.
+ * This fixes the bug where array((1L,2L,3L))*2 sometimes dumps core
+ * However the ternary versions of PyNumber_Power should be properly
+ * supported, once this gets done we can eliminate this hack and also
+ * get the ternary version of pow() to work for arrays */
+static PyObject * PyNumber_PowerBinary(PyObject *v,PyObject *w){
+ return PyNumber_Power(v,w,Py_None);
+}
static PyUFuncGenericFunction add_functions[] = { UBYTE_add, SBYTE_add, SHORT_add, INT_add, LONG_add, FLOAT_add, DOUBLE_add, CFLOAT_add, CDOUBLE_add, NULL, };
static PyUFuncGenericFunction subtract_functions[] = { UBYTE_subtract, SBYTE_subtract, SHORT_subtract, INT_subtract, LONG_subtract, FLOAT_subtract, DOUBLE_subtract, CFLOAT_subtract, CDOUBLE_subtract, NULL, };
@@ -1749,5 +1758,5 @@
static void * conjugate_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)"conjugate", };
static void * remainder_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)fmod, (void *)fmod, (void *)PyNumber_Remainder, };
-static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_Power, };
+static void * power_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)pow, (void *)pow, (void *)c_pow, (void *)c_pow, (void *)PyNumber_PowerBinary, };
static void * absolute_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Absolute, };
static void * negative_data[] = { (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)NULL, (void *)PyNumber_Negative, };