numpy vs numeric benchmarks
Hello! I've been using numeric for a while, and the recent list traffic prompted me to finally migrate all my old code. On a whim, we were benchmarking numpy vs numeric and have been lead to the conclusion that numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy but 300 ms in numeric. Now, of course, I don't believe this, but I can't figure out what we're doing wrong; I'm not the only person who has looked at this code, so can anyone tell me what we're doing wrong? We run both benchmarks twice to try and mitigate any start-up and cache effects. This is with debian-amd64's packaged numeric 24.2-2 and a locally built numpy-0.9.8. /usr/bin/python import time import numpy import random import Numeric def numpytest(): N = 1000 x = numpy.zeros((N,N),'f') y = numpy.zeros((N,N),'f') for i in range(N): for j in range(N): x[i, j] = random.random() y[i, j] = random.random() t1 = time.clock() z = numpy.matrixmultiply(x, y) t2 = time.clock() print (((t2 - t1)*1000)) def numerictest(): N = 1000 x = Numeric.zeros((N,N),'f') y = Numeric.zeros((N,N),'f') for i in range(N): for j in range(N): x[i, j] = random.random() y[i, j] = random.random() t1 = time.clock() z = Numeric.matrixmultiply(x, y) t2 = time.clock() print (((t2 - t1)*1000)) numerictest() numpytest() numpytest() numerictest() on our hardware a call to numerictest() takes 340 ms and a numpytest takes around 13 sec (!). Any advice on what we're doing wrong would be very helpful. ...Eric
On Friday 02 June 2006 14:58, Eric Jonas wrote: [EJ]: Hello! I've been using numeric for a while, and the recent list traffic [EJ]: prompted me to finally migrate all my old code. On a whim, we were [EJ]: benchmarking numpy vs numeric and have been lead to the conclusion that [EJ]: numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy [EJ]: but 300 ms in numeric. You mean the other way around? I also tested numpy vs numarray, and numarray seems to be roughly 3 times faster than numpy for your particular testcase. J. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
I meant "numeric is slower than numpy", that is, modern numpy (0.9.8) appears to lose out majorly to numeric. This doesn't make much sense, so I figured there was something wrong with my benchmark, or my numpy install, and wanted to check if others had seen this sort of behavior. ...Eric On Fri, 2006-06-02 at 15:27 +0200, Joris De Ridder wrote:
On Friday 02 June 2006 14:58, Eric Jonas wrote: [EJ]: Hello! I've been using numeric for a while, and the recent list traffic [EJ]: prompted me to finally migrate all my old code. On a whim, we were [EJ]: benchmarking numpy vs numeric and have been lead to the conclusion that [EJ]: numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy [EJ]: but 300 ms in numeric.
You mean the other way around?
I also tested numpy vs numarray, and numarray seems to be roughly 3 times faster than numpy for your particular testcase.
J.
Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Joris De Ridder wrote:
On Friday 02 June 2006 14:58, Eric Jonas wrote: [EJ]: Hello! I've been using numeric for a while, and the recent list traffic [EJ]: prompted me to finally migrate all my old code. On a whim, we were [EJ]: benchmarking numpy vs numeric and have been lead to the conclusion that [EJ]: numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy [EJ]: but 300 ms in numeric.
You mean the other way around?
I also tested numpy vs numarray, and numarray seems to be roughly 3 times faster than numpy for your particular testcase.
Please post your test cases. We are trying to remove any slowness, but need testers to do it. -Travis
Hi, It seems that in Numeric the matrixmultiply is alias for dot function, which "uses the BLAS optimized routines where possible", as the help() says. In NumPy (0.9.6, not upgraded yet to 0.9.8), the matrixmultiply is a function of numpy.core.multiarray, while dot refers to numpy.core._dotblas. On my system the timings and results with numpy.dot are quite similar to that with Numeric.matrixmultiply. So the next question is what's the difference between matrixmultiply and dot in NumPy? Filip
Hello! I've been using numeric for a while, and the recent list traffic prompted me to finally migrate all my old code. On a whim, we were benchmarking numpy vs numeric and have been lead to the conclusion that numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy but 300 ms in numeric.
Now, of course, I don't believe this, but I can't figure out what we're doing wrong; I'm not the only person who has looked at this code, so can anyone tell me what we're doing wrong?
Yes, using numpy.dot I get 250ms, numpy.matrixmultiply 11.8s. while a sans-BLAS Numeric.matrixmultiply takes 12s. The first 100 results from numpy.dot and numpy.matrixmultiply are identical .... Use dot;) --George. On 02/06/06, Filip Wasilewski <filip@ftv.pl> wrote:
Hi,
It seems that in Numeric the matrixmultiply is alias for dot function, which "uses the BLAS optimized routines where possible", as the help() says.
In NumPy (0.9.6, not upgraded yet to 0.9.8), the matrixmultiply is a function of numpy.core.multiarray, while dot refers to numpy.core._dotblas.
On my system the timings and results with numpy.dot are quite similar to that with Numeric.matrixmultiply.
So the next question is what's the difference between matrixmultiply and dot in NumPy?
Filip
Hello! I've been using numeric for a while, and the recent list traffic prompted me to finally migrate all my old code. On a whim, we were benchmarking numpy vs numeric and have been lead to the conclusion that numpy is at least 50x slower; a 1000x1000 matmul takes 16 sec in numpy but 300 ms in numeric.
Now, of course, I don't believe this, but I can't figure out what we're doing wrong; I'm not the only person who has looked at this code, so can anyone tell me what we're doing wrong?
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Just to be sure, what exactly is affected when one uses the slower algorithms when neither BLAS or LAPACK is installed? For sure it will affect almost every function in numpy.linalg, as they use LAPACK_lite. And I guess that in numpy.core the dot() function uses the lite numpy/core/blasdot/_dotblas.c routine? Any other numpy functions that are affected? Joris On Friday 02 June 2006 16:16, George Nurser wrote: [GN]: Yes, using numpy.dot I get 250ms, numpy.matrixmultiply 11.8s. [GN]: [GN]: while a sans-BLAS Numeric.matrixmultiply takes 12s. [GN]: [GN]: The first 100 results from numpy.dot and numpy.matrixmultiply are identical .... [GN]: [GN]: Use dot;) [GN]: [GN]: --George. Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm
Joris De Ridder wrote:
Just to be sure, what exactly is affected when one uses the slower algorithms when neither BLAS or LAPACK is installed? For sure it will affect almost every function in numpy.linalg, as they use LAPACK_lite. And I guess that in numpy.core the dot() function uses the lite numpy/core/blasdot/_dotblas.c routine? Any other numpy functions that are affected?
convolve could also be affected (the basic internal _dot function gets replaced for FLOAT, DOUBLE, CFLOAT, and CDOUBLE). I think that's the only function that uses dot internally. In the future we hope to be optimizing ufuncs as well. -Travis
Is there some way, either within numpy or at build-time, to verify you're using BLAS/LAPACK? Is there one we should be using? ...Eric On Fri, 2006-06-02 at 11:19 -0600, Travis Oliphant wrote:
Joris De Ridder wrote:
Just to be sure, what exactly is affected when one uses the slower algorithms when neither BLAS or LAPACK is installed? For sure it will affect almost every function in numpy.linalg, as they use LAPACK_lite. And I guess that in numpy.core the dot() function uses the lite numpy/core/blasdot/_dotblas.c routine? Any other numpy functions that are affected?
convolve could also be affected (the basic internal _dot function gets replaced for FLOAT, DOUBLE, CFLOAT, and CDOUBLE). I think that's the only function that uses dot internally.
In the future we hope to be optimizing ufuncs as well.
-Travis
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Fri, 2 Jun 2006 19:09:01 +0200 Joris De Ridder <joris@ster.kuleuven.be> wrote:
Just to be sure, what exactly is affected when one uses the slower algorithms when neither BLAS or LAPACK is installed? For sure it will affect almost every function in numpy.linalg, as they use LAPACK_lite. And I guess that in numpy.core the dot() function uses the lite numpy/core/blasdot/_dotblas.c routine? Any other numpy functions that are affected?
Using a better default dgemm for matrix multiplication when an optimized BLAS isn't available has been on my to-do list for a while. I think it can be speed up by a large amount on a generic machine by using blocking of the matrices. Personally, I perceive no difference between my g77-compiled LAPACK, and the gcc-compiled f2c'd routines in lapack_lite, if an optimized BLAS is used. And lapack_lite has fewer bugs than the version of LAPACK available off of netlib.org, as I used the latest patches I could scrounge up (mostly from Debian). -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
Filip Wasilewski wrote:
So the next question is what's the difference between matrixmultiply and dot in NumPy?
matrixmultiply is a deprecated compatibility name. Always use dot. dot will get replaced with the optimized dotblas implementation when an optimized BLAS is available. matrixmultiply will not (probably not intentionally, but I'm happy with the current situation). -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Robert Kern wrote:
Filip Wasilewski wrote:
So the next question is what's the difference between matrixmultiply and dot in NumPy?
matrixmultiply is a deprecated compatibility name. Always use dot. dot will get replaced with the optimized dotblas implementation when an optimized BLAS is available. matrixmultiply will not (probably not intentionally, but I'm happy with the current situation).
It's true that matrixmultiply has been deprecated for some time (at least 8 years...) The basic dot function gets over-written with a BLAS-optimized version but the matrixmultiply does not get changed. So replace matrixmultiply with dot. It wasn't an intentional thing, but perhaps it will finally encourage people to always use dot. -Travis
A Divendres 02 Juny 2006 19:07, Travis Oliphant va escriure:
Robert Kern wrote:
Filip Wasilewski wrote:
So the next question is what's the difference between matrixmultiply and dot in NumPy?
matrixmultiply is a deprecated compatibility name. Always use dot. dot will get replaced with the optimized dotblas implementation when an optimized BLAS is available. matrixmultiply will not (probably not intentionally, but I'm happy with the current situation).
It's true that matrixmultiply has been deprecated for some time (at least 8 years...) The basic dot function gets over-written with a BLAS-optimized version but the matrixmultiply does not get changed. So replace matrixmultiply with dot. It wasn't an intentional thing, but perhaps it will finally encourage people to always use dot.
So, why not issuing a DeprecationWarning on a matrixmultiply function use? --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
participants (8)
-
David M. Cooke -
Eric Jonas -
Filip Wasilewski -
Francesc Altet -
George Nurser -
Joris De Ridder -
Robert Kern -
Travis Oliphant