matrix multipln takes too much time
hi i am doing some maths calculations involving matrices of double values using numpy.matrix , java code for this is something like int items=25; int sample=5; int totalcols=8100; double[][]dblarrayone=new double[items][totalcols]; double[][]dblarraytwo=new double[items][totalcols]; //their elements are set elsewhere before calculation double[][] resultarray = new double[items][sample]; for(int i=0;i<items;i++){ for(int j=0;j<sample;j++){ double tval=0.0; for(int p=0;p<totalcols;p++) tval+=dblarrayone[j][p] * dblarraytwo[i][p]; resultarray[i][j]=Math.abs(tval); } } so I wanted to do the same in python..(may be this code is not in the recommended way..) i am storing the filled matrices and other values as instance variable of a class and access them by self.whatever... self.items=25 self.sample=5 self.totalcols=8100 #self.matrixone,self.matrixtwo are numply matix objects with already filled elements #but for testing i filled it with zeros self.matrixone=matrix(zeros((items,totalcols))) self.matrixtwo=matrix(zeros((items,totalcols))) resultmatrix=matrix(zeros((self.items,self.sample))) for i in range(self.items): for j in range(self.sample): tval=0.0 for p in range(self.totalcols): tval +=self.matrixone[ j , p ] * self.matrixtwo[ i , p ] resultmatrix[ i, j ]=abs(tval) here I found that while the java code takes barely 110 milliseconds to execute the code ,the python code takes something like 53 secs to execute !!..I am baffled by this ..can anyone advise me how i can improve this? (i want to code in python so I can't use c,c++ , java) dn
Hi there - quick suggestion on Xmas morning - others are much more familar. You do not want to use a loop to do the matrix multiply, you want to use the intrinsic functions assoicated with matrix. So you want something like res = Math.abs( matmul(arrayone, arraytwo) ) note - that is not real code, just symbolic code. I am sure this is easily found in the documentation. cheers! Lou On Dec 25, 2007, at 8:47 AM, devnew@gmail.com wrote:
hi i am doing some maths calculations involving matrices of double values using numpy.matrix ,
java code for this is something like
int items=25; int sample=5; int totalcols=8100; double[][]dblarrayone=new double[items][totalcols]; double[][]dblarraytwo=new double[items][totalcols]; //their elements are set elsewhere before calculation
double[][] resultarray = new double[items][sample];
for(int i=0;i<items;i++){ for(int j=0;j<sample;j++){ double tval=0.0; for(int p=0;p<totalcols;p++) tval+=dblarrayone[j][p] * dblarraytwo[i][p]; resultarray[i][j]=Math.abs(tval);
}
}
so I wanted to do the same in python..(may be this code is not in the recommended way..) i am storing the filled matrices and other values as instance variable of a class and access them by self.whatever...
self.items=25 self.sample=5 self.totalcols=8100 #self.matrixone,self.matrixtwo are numply matix objects with already filled elements #but for testing i filled it with zeros self.matrixone=matrix(zeros((items,totalcols))) self.matrixtwo=matrix(zeros((items,totalcols))) resultmatrix=matrix(zeros((self.items,self.sample)))
for i in range(self.items): for j in range(self.sample): tval=0.0 for p in range(self.totalcols): tval +=self.matrixone[ j , p ] * self.matrixtwo[ i , p ] resultmatrix[ i, j ]=abs(tval)
here I found that while the java code takes barely 110 milliseconds to execute the code ,the python code takes something like 53 secs to execute !!..I am baffled by this ..can anyone advise me how i can improve this? (i want to code in python so I can't use c,c++ , java)
dn _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
---------------------------------------------------------------------------- | Dr. Louis J. Wicker | NSSL/WRDD Rm 4366 | National Weather Center | 120 David L. Boren Boulevard, Norman, OK 73072 | | E-mail: Louis.Wicker@noaa.gov | HTTP: www.nssl.noaa.gov/~lwicker | Phone: (405) 325-6340 | Fax: (405) 325-6780 | | "Programming is not just creating strings of instructions | for a computer to execute. It's also 'literary' in that you | are trying to communicate a program structure to | other humans reading the code." - Paul Rubin | |"Real efficiency comes from elegant solutions, not optimized programs. | Optimization is always just a few correctness-preserving transformations | away." - Jonathan Sobel ---------------------------------------------------------------------------- | | "The contents of this message are mine personally and | do not reflect any position of the Government or NOAA." | ----------------------------------------------------------------------------
you can either use matrix multiplication (see resultmatrix2) or tensordot (see resultmatrix3). on my computer I have: 1. 15.6 sec with your code 2. 0.072 sec with resultmatrix2 3. 0.040 sec with tensordot (resultmatrix3) (-- which is a 400x speed) -------------------------------------------- from numpy import * items=25 sample=5 totalcols=8100 #matrixone=matrix(zeros((items,totalcols))) #matrixtwo=matrix(zeros((items,totalcols))) matrixone=matrix(random.rand(items, totalcols)) matrixtwo=matrix(random.rand(items, totalcols)) resultmatrix=matrix(zeros((items,sample))) resultmatrix2=matrix(zeros((items,sample))) resultmatrix3=matrix(zeros((items,sample))) # your code for i in range(items): for j in range(sample): tval=0.0 for p in range(totalcols): tval +=matrixone[ j , p ] * matrixtwo[ i , p ] resultmatrix[ i, j ]=abs(tval) # matrix multiplication for i in range(items): for j in range(sample): resultmatrix2[ i, j ] = abs(matrixone[j,:] * matrixtwo[i,:].T) # tensordot resulmatrix3 = tensordot(matrixone[:sample,:], matrixtwo.T, axes=1).T --------------------------------------- hth, L. On Dec 25, 2007 5:16 PM, Louis Wicker <Louis.Wicker@noaa.gov> wrote:
Hi there - quick suggestion on Xmas morning - others are much more familar. You do not want to use a loop to do the matrix multiply, you want to use the intrinsic functions assoicated with matrix.
So you want something like
res = Math.abs( matmul(arrayone, arraytwo) )
note - that is not real code, just symbolic code. I am sure this is easily found in the documentation.
cheers!
Lou
On Dec 25, 2007, at 8:47 AM, devnew@gmail.c om <devnew@gmail.com> w le-interchange-newline">
hi i am doing some maths calculations involving matrices of double values using numpy.matrix ,
java code for this is something like
int items=25; int sample=5; int totalcols=8100; double[][]dblarrayone=new double[items][totalcols]; double[][]dblarraytwo=new double[items][totalcols]; //their elements are set elsewhere before calculation
double[][] resultarray = new double[items][sample];
for(int i=0;i<items;i++){ for(int j=0;j<sample;j++){ double tval=0.0; for(int p=0;p<totalcols;p++) tval+=dblarrayone[j][p] * dblarraytwo[i][p]; resultarray[i][j]=Math.abs(tval);
}
}
so I wanted to do the same in python ..(may b recommended way..) i am storing the filled matrices and other values as instance variable of a class and access them by self.whatever...
self.items=25 self.sample=5 self.totalcols=8100 #self.matrixone,self.matrixtwo are numply matix objects with already filled elements #but for testing i filled it with zeros self.matrixone=matrix(zeros((items,totalcols))) self.matrixtwo=matrix(zeros((items,totalcols))) resultmatrix=matrix(zeros((self.items,self.sample)))
for i in range(self.items): for j in range(self.sample): tval=0.0 for p in range(self.totalcols): tval +=self.matrixone[ j , p ] * self.matrixtwo[ i , p ] resultmatrix[ i, j ]=abs(tval)
here I found that while the java code takes ba br>execute the code ,the python code takes something like 53 secs to execute !!..I am baffled by this ..can anyone advise me how i can improve this? (i want to code in python so I can't use c,c++ , java)
dn _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
| Dr. Louis J. Wicker | NSSL/WRDD Rm 4366 | National Weather Center < "Apple-style-span" style="line-height: 16px;; font-size: 14px; "> | 120 David L. Boren Boulevard, Norman, OK 73072 | | E-mail: Louis.Wicker@noaa.gov | HTTP: www.nssl.noaa.gov/~lwicker <http://www.nssl.noaa.gov/%7Elwicker> | Phone: (405) 325-6340 | Fax: (405) 325-6780 | | "Programming is not just creating strings of instructions | for a computer to execute. It's also 'literary' in that you | are trying to communicate a program structure to | other humans reading the code." - Paul Rubin | |"Real efficiency comes from elegant solutions, not optimiz ed progr le="font-size: 14px; ">| Optimization is always just a few correctness-preserving transformations | away." - Jonathan Sobel
---------------------------------------------------------------------------- | | "The contents of this message are mine personally and | do not reflect any position of the Government or NOAA." |
----------------------------------------------------------------------------
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
on my computer I have: 1. 15.6 sec with your code 2. 0.072 sec with resultmatrix2 3. 0.040 sec with tensordot (resultmatrix3) (-- which is a 400x speed)
wow ,thanks! the tensordot fn is blinding fast.. i added /modified resultndarray = tensordot(matrixone[:sample,:], matrixtwo.T, axes=1).T resultmatrix =matrix(abs(resultndarray)) so i can get rid of the negative values in a real case ..your replies helped a lot guys,thanx a lot and happy X'Mas dn
participants (3)
-
devnew@gmail.com
-
lorenzo bolla
-
Louis Wicker