On Tue, Dec 22, 2009 at 12:40 PM, Pauli Virtanen <pav@iki.fi> wrote:
ti, 2009-12-22 kello 10:16 -0700, Charles R Harris kirjoitti: [clip: PyOS_ascii_strtod -> PyOS_string_to_double]
The patch looks ok, but the functions handle errors differently and I wonder if that has been completely audited.
It can actually still crash from the same reason: PyOS_string_to_double docs say:
"""If no initial segment of the string is the valid representation of a floating-point number, set *endptr to point to the beginning of the string, raise ValueError, and return -1.0"""
Indeed,
$ gdb --args python3 -c "import numpy as np; np.fromstring('1,,', sep=',')" (gdb) run Program received signal SIGSEGV, Segmentation fault. PyErr_SetObject (exception=0x8291740, value=0xb7e926a0) at ../Python/errors.c:67 67 ../Python/errors.c: Tiedostoa tai hakemistoa ei ole. in ../Python/errors.c (gdb) bt #0 PyErr_SetObject (exception=0x8291740, value=0xb7e926a0) at ../Python/errors.c:67 #1 0x080e8d5a in PyErr_Format (exception=0x8291740, format=0x81a0998 "could not convert string to float: %.200s") at ../Python/errors.c:638 #2 0x080fb5fe in PyOS_string_to_double (s=0xb7ca2ae2 ",", endptr=0xbfffd130, overflow_exception=0x0) at ../Python/pystrtod.c:354 #3 0x004a9bfc in NumPyOS_ascii_strtod (s=0xb7ca2ae2 ",", endptr=0xbfffd130) at numpy/core/src/multiarray/numpyos.c:525
I suppose raising an exception requires ownership of GIL. So either we implement ASCII number parsing ourselves from scratch (or steal it from somewhere), or surround the call with appropriate GIL-acquiring wrappers plus if (PyErr_Occurred()) PyErr_Clear();
Could you expand a bit on this? There are several places where PyErr_Occurred are called and I am wondering if there is a problem. In fact, I moved one such check and a segfault went away, which made me suspicious... Chuck