[Numpy-discussion] Error when multiplying large sparse matrices
Jeremy Lecoeur
jeremy.lecoeur at uphs.upenn.edu
Fri Jun 1 14:45:04 EDT 2012
Hi,
I have been using the sparse matrix tools for a while to do all sort of
things and, using the same code that was working just fine, I now
encounter a problem when trying . I do have very large sparse matrices
and when i multiplying them the number of non zeros exceed the max value
of an intc, which cause indptr to hold negative values. Hence in the
multiplication function of csr, whenc reating the resulting matrix, i
get an error as it is not possible to have a negative value for a matrix
size.
Am I missing something that would allow me to do that computation ?
Here is the code I am using:
def main():
inCondMatFile = sys.argv[1]
inNodeSize = sys.argv[2]
outProfileFile = sys.argv[3]
outNodeDistFile = sys.argv[4]
outNodeDensityFile = sys.argv[5]
Acsr = scipy.io.mmread(inCondMatFile).tocsr().sorted_indices()
A = Acsr.tocoo()
n = A.shape[0]
nnz = A.nnz
rows=numpy.zeros(3*nnz+n, dtype=numpy.int32)
cols=numpy.zeros(3*nnz+n, dtype=numpy.int32)
data=numpy.zeros(3*nnz+n, dtype=numpy.float64)
#first n rows of constraint mat is A - I
rows[0:nnz] = A.row
cols[0:nnz] = A.col
data[0:nnz] = A.data
rows[nnz:nnz+n] = numpy.arange(n)
cols[nnz:nnz+n] = numpy.arange(n)
data[nnz:nnz+n] = -numpy.ones(n)
#rows n to n+nnz are
#A_{i,j} d_{j} - A_{j,i} d_{i} == 0
rows[nnz+n:] = numpy.append(numpy.arange(n,n+nnz),numpy.arange(n,n+nnz))
cols[nnz+n:] = numpy.append(A.col,A.row)
data[nnz+n:] = numpy.append(A.data,-Acsr[A.col,A.row])
tmpC = scipy.sparse.coo_matrix( (data, (rows,cols) ) )
Ptmp = (tmpC.transpose().tocsr() * tmpC.tocsr()).tocoo()
And it fails for that last multiplication (i did not include the rest of
the code) because of an nnz way too big for an intc.
Jeremy
The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
More information about the NumPy-Discussion
mailing list