On 12/15/2009 10:59 AM, Charles R Harris wrote:


On Tue, Dec 15, 2009 at 9:51 AM, Pauli Virtanen <pav+sp@iki.fi> wrote:
Tue, 15 Dec 2009 10:36:03 -0600, Bruce Southey wrote:
[clip]
> Program received signal SIGSEGV, Segmentation fault. setup_context
> (registry=<value optimized out>, module=<value optimized out>,
> lineno=<value optimized out>, filename=<value optimized out>,
> stack_level=<value optimized out>)
>      at Python/_warnings.c:449
> 449         PyFrameObject *f = PyThreadState_GET()->frame; (gdb) bt
> #0  setup_context (registry=<value optimized out>, module=<value
> optimized out>, lineno=<value optimized out>, filename=<value optimized
> out>, stack_level=<value optimized out>)
>      at Python/_warnings.c:449
> #1  do_warn (registry=<value optimized out>, module=<value optimized
> out>, lineno=<value optimized out>, filename=<value optimized out>,
> stack_level=<value optimized out>)
>      at Python/_warnings.c:593
> #2  0x0000000000493c81 in PyErr_WarnEx (category=0x760720, text=<value
> optimized out>, stack_level=1) at Python/_warnings.c:719 #3
> 0x00000000004c8e94 in PyOS_ascii_strtod (nptr=0x7ffff7f08914 "1 , 2 , 3
> , 4", endptr=0x7fffffffdb28) at Python/pystrtod.c:282 #4
> 0x00007ffff2954151 in NumPyOS_ascii_strtod (s=0x7ffff7f08914 "1 , 2 , 3
> , 4", endptr=0x7fffffffdb28) at numpy/core/src/multiarray/numpyos.c:527

Looks like it's trying to raise a deprecation warning after
PyArray_FromString has released GIL. So that was the reason why it caused
a segfault also in 3.1.

PyOS_ascii_strtod was deprecated in 2.7 and in 3.1. Apparently, we now
*must* do something like

#if PY_VERSION_HEX >= 0x02060000
   return PyOS_string_to_double(s, endptr, NULL);
#else
   return PyOS_ascii_strtod(s, endptr);
#endif

everywhere the function is used.

It also seems that this needs to be backported to Numpy 1.4.x...

(Note to self: this is also the origin of the crashes in scipy/
lambertw... GIL must be reacquired before raising any warnings.)


Would it be appropriate to put macros for all these in config.h or some other common spot? Having all the python version dependencies in one spot might make it easier to keep current. I've been thinking of moving the numpy deprecation macro for that reason.

Chuck

There are two places that appear to use PyOS_ascii_strtod in numpy:
1) core/SConscript -  'Define the function PyOS_ascii_strod if not available'
There is no definition for Python 3K and I do not know the purpose of this function.

2)  core/src/multiarray/numpyos.c - defines NumPyOS_ascii_strtod:
Here the usage is already within an '#if defined(NPY_PY3K)'  for Python 3K. In the numpyos.c with numpy '1.5.0.dev8019' I replaced the NPY_PY3K with:
#if PY_VERSION_HEX >= 0x02070000

In Python 2.6 all the tests passed. But for Python 2.7 I got the following errors.

======================================================================
ERROR: test_buffer_hashlib (test_regression.TestRegression)          
----------------------------------------------------------------------
Traceback (most recent call last):                                   
  File "/usr/local/lib/python2.7/site-packages/numpy/core/tests/test_regression.py", line 1255, in test_buffer_hashlib
    assert_equal(md5(x).hexdigest(), '2a1dd1e1e59d0a384c26951e316cd7e6')                                             
TypeError: object supporting the buffer API required                                                                 

======================================================================
FAIL: Check formatting of complex types.                             
----------------------------------------------------------------------
Traceback (most recent call last):                                   
  File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)                                                         
  File "/usr/local/lib/python2.7/site-packages/numpy/core/tests/test_print.py", line 61, in check_complex_type
    err_msg='Failed str formatting for type %s' % tp)
  File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 305, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed str formatting for type <type 'numpy.complex64'>
 ACTUAL: '-1j'
 DESIRED: '(-0-1j)'

======================================================================
FAIL: Check formatting of complex types.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/usr/local/lib/python2.7/site-packages/numpy/core/tests/test_print.py", line 61, in check_complex_type
    err_msg='Failed str formatting for type %s' % tp)
  File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 305, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed str formatting for type <type 'numpy.complex128'>
 ACTUAL: '-1j'
 DESIRED: '(-0-1j)'

======================================================================
FAIL: Check formatting of complex types.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/usr/local/lib/python2.7/site-packages/numpy/core/tests/test_print.py", line 61, in check_complex_type
    err_msg='Failed str formatting for type %s' % tp)
  File "/usr/local/lib/python2.7/site-packages/numpy/testing/utils.py", line 305, in assert_equal
    raise AssertionError(msg)
AssertionError:
Items are not equal: Failed str formatting for type <type 'numpy.complex256'>
 ACTUAL: '-1j'
 DESIRED: '(-0-1j)'

----------------------------------------------------------------------
Ran 2494 tests in 6.932s

FAILED (KNOWNFAIL=5, errors=1, failures=3)
<nose.result.TextTestResult run=2494 errors=1 failures=3>

These last ones may be due to:

"Another format()-related change: the default precision used for floating-point and complex numbers was changed from 6 decimal places to 12, which matches the precision used by str(). (Changed by Eric Smith; issue 5920.)"
http://docs.python.org/dev/whatsnew/2.7.html

Bruce