Hi all, I am going to visualize the sparsity pattern of large matrices using numpy/scipy/matplotlib. I received a segfault running the following script from pylab import show, plot, axis, xlabel, subplot, title, spy from scipy import * K = io.mmread('k.mtx') M = io.mmread('m.mtx') print 'Reading finished' K = K.todense() spy(K) show() Starting program: /usr/bin/python matrixstructure.py [Thread debugging using libthread_db enabled] [New Thread 16384 (LWP 13423)] Reading finished Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 16384 (LWP 13423)] array_from_pyobj (type_num=12, dims=0x7fffffffd270, rank=2, intent=0, obj=<value optimized out>) at fortranobject.c:547 547 PyArray_FILLWBYTE(arr, 0); Current language: auto; currently c (gdb) bt #0 array_from_pyobj (type_num=12, dims=0x7fffffffd270, rank=2, intent=0, obj=<value optimized out>) at fortranobject.c:547 #1 0x00002aaab37606ab in f2py_rout_sparsetools_dcsctofull (capi_self=<value optimized out>, capi_args=<value optimized out>, capi_keywds=<value optimized out>, f2py_func=0x2aaab3778760 <dcsctofull_>) at sparsetoolsmodule.c:12056 #2 0x00002aaab376f25b in fortran_call (fp=<value optimized out>, arg=<value optimized out>, kw=<value optimized out>) at fortranobject.c:289 #3 0x00002aaaaabf7f20 in PyObject_Call () from /usr/lib64/libpython2.4.so.1.0 #4 0x00002aaaaac4ebaa in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0 #5 0x00002aaaaac510af in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0 #6 0x00002aaaaac510af in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0 #7 0x00002aaaaac510af in PyEval_EvalFrame () from /usr/lib64/libpython2.4.so.1.0 #8 0x00002aaaaac51705 in PyEval_EvalCodeEx () from /usr/lib64/libpython2.4.so.1.0 #9 0x00002aaaaac51992 in PyEval_EvalCode () from /usr/lib64/libpython2.4.so.1.0 #10 0x00002aaaaac6aeb9 in run_node () from /usr/lib64/libpython2.4.so.1.0 #11 0x00002aaaaac6c3c9 in PyRun_SimpleFileExFlags () from /usr/lib64/libpython2.4.so.1.0 #12 0x00002aaaaac6c921 in PyRun_AnyFileExFlags () from /usr/lib64/libpython2.4.so.1.0 #13 0x00002aaaaac72023 in Py_Main () from /usr/lib64/libpython2.4.so.1.0 #14 0x00000000004008d9 in main (argc=<value optimized out>, argv=<value optimized out>) at ccpython.cc:10 Any idea how to fix this problem ? Nils
Nils Wagner wrote:
Hi all,
I am going to visualize the sparsity pattern of large matrices using numpy/scipy/matplotlib. I received a segfault running the following script
from pylab import show, plot, axis, xlabel, subplot, title, spy from scipy import *
K = io.mmread('k.mtx') M = io.mmread('m.mtx') print 'Reading finished'
K = K.todense() spy(K) show()
Thanks for the traceback. It looks like an error was not being caught. This is now fixed in SVN f2py. Your code should now raise an error that is occurring in trying to convert an object to an array in mmread... -Travis
Travis Oliphant wrote:
Nils Wagner wrote:
Hi all,
I am going to visualize the sparsity pattern of large matrices using numpy/scipy/matplotlib. I received a segfault running the following script
from pylab import show, plot, axis, xlabel, subplot, title, spy from scipy import *
K = io.mmread('k.mtx') M = io.mmread('m.mtx') print 'Reading finished'
K = K.todense() spy(K) show()
Thanks for the traceback. It looks like an error was not being caught. This is now fixed in SVN f2py. Your code should now raise an error that is occurring in trying to convert an object to an array in mmread...
-Travis
Reading finished Traceback (most recent call last): File "matrixstructure.py", line 8, in ? K = K.todense() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 356, in todense return asmatrix(self.toarray()) File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 360, in toarray return csc.toarray() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 969, in toarray return func(self.shape[0], self.data, self.rowind, self.indptr) MemoryError Nils
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev
Nils Wagner wrote:
Reading finished Traceback (most recent call last): File "matrixstructure.py", line 8, in ? K = K.todense() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 356, in todense return asmatrix(self.toarray()) File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 360, in toarray return csc.toarray() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 969, in toarray return func(self.shape[0], self.data, self.rowind, self.indptr) MemoryError
Notabug. The array is simply too large to be represented as a dense array. You need to either get more memory, or write a procedure to plot the sparsity pattern without creating the dense version of the array. Good luck. -- Robert Kern robert.kern@gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Nils Wagner wrote:
Travis Oliphant wrote:
Nils Wagner wrote:
Hi all,
I am going to visualize the sparsity pattern of large matrices using numpy/scipy/matplotlib. I received a segfault running the following script
from pylab import show, plot, axis, xlabel, subplot, title, spy from scipy import *
K = io.mmread('k.mtx') M = io.mmread('m.mtx') print 'Reading finished'
K = K.todense() spy(K) show()
Thanks for the traceback. It looks like an error was not being caught. This is now fixed in SVN f2py. Your code should now raise an error that is occurring in trying to convert an object to an array in mmread...
-Travis
Reading finished Traceback (most recent call last): File "matrixstructure.py", line 8, in ? K = K.todense() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 356, in todense return asmatrix(self.toarray()) File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 360, in toarray return csc.toarray() File "/usr/lib64/python2.4/site-packages/scipy/sparse/sparse.py", line 969, in toarray return func(self.shape[0], self.data, self.rowind, self.indptr) MemoryError
Could you use pdb import pdb pdb.pm() to find out what self.shape[0] is? This will help track down the problem. -Travis
participants (4)
-
Nils Wagner -
Robert Kern -
Travis Oliphant -
Travis Oliphant