Re: [SciPy-Dev] Baffling error: ndarray += csc_matrix -> "ValueError: setting an array element with a sequence"
[CC'ing scipy-dev because see below] On Tue, Sep 24, 2013 at 6:45 PM, Nathaniel Smith <njs@pobox.com> wrote:
Hi all,
I'm getting a very strange error, and unfortunately I can't seem to reproduce it *except* when running on Travis-CI, so my debugging tools are really limited. I'm wondering if anyone else has seen anything like this?
The offending line of code is:
File "/home/travis/virtualenv/python2.7_with_system_site_packages/local/lib/python2.7/site-packages/pyrerp/incremental_ls.py", line 323, in append_bottom_half self.xtx += xtx ValueError: setting an array element with a sequence.
And debug prints reveal that in the case that causes the error, 'self.xtx' is an ndarray that prints as:
[[ 0. 0.] [ 0. 0.]]
while 'xtx' is a scipy.sparse.csc.csc_matrix that prints as:
(1, 0) 45.0 (0, 0) 10.0 (1, 1) 285.0 (0, 1) 45.0
In accordance with the cosmic law governing such things, this turns out to be a bug in numpy that I introduced in 2397c9d4, specifically this line, which introduces an unchecked DEPRECATE: https://github.com/numpy/numpy/commit/2397c9d4#L5R528 (Plus some complicated nonsense involving virtualenvs that sometimes fall back on system libraries even though the virtualenv contains a perfectly good version of the same library etc. to make reproduction more confusing.) My test code was setting warnings to raise errors by default, and apparently ndarray += csc_matrix goes through some circuitous path that (AFAICT) involves creating an object ndarray containing the csc_matrix and calling the add ufunc, which somehow trips on the cast warning above. Then later on, at line 159 of arraytypes.c.src, the the @TYPE@_setitem function does: if (PyErr_Occurred()) { if (PySequence_Check(op) && !PyString_Check(op) && !PyUnicode_Check(op)) { PyErr_Clear(); PyErr_SetString(PyExc_ValueError, "setting an array element with a sequence."); } return -1; } and the PyErr_Occurred here catches the real error and replaces it with some nonsense. So three points: 1) Maybe this will be useful to someone googling later. 2) I guess I'll file some horrible patch against numpy that just throws away exceptions caused by that deprecation, because there is no way for PyArray_CanCastTypeTo to raise an error :-(. 3) This script raises a DeprecationWarning with numpy 1.7.1 and scipy 0.12.0: import warnings, numpy, scipy.sparse warnings.filterwarnings("always") a = numpy.zeros((2, 2)) b = scipy.sparse.csc_matrix([[0.0, 0.0], [0.0, 0.0]]) a += b test-script3.py:5: DeprecationWarning: Implicitly casting between incompatible kinds. In a future numpy release, this will raise an error. Use casting="unsafe" if this is intentional. I really don't understand what arcane magic is used to make ndarray += csc_matrix work at all, but my question is, is it going to break when we complete the casting transition described above? It was just supposed to catch things like int += float. -n
participants (1)
-
Nathaniel Smith