# NumericNumarrayTest.py ''' A test, provided by Francesc Altet http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890 ''' from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) # scipy/Numeric3 added setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) RESULTS:
pythonw -u "NumericNumarrayTest.py" Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899 Exit code: 0
Colin W.
A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure:
RESULTS:
pythonw -u "NumericNumarrayTest.py"
Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899
Is that really true? From my original post:
from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 0.12782907485961914 setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 1.2013700008392334
My post was from January, so, in the interim numarray has improved *a lot* it's object creation time. In that case, why the numarray team hasn't publisized that more? Well, I think I should be missing something. Time for nap. Cheers, --
0,0< Francesc Altet http://www.carabos.com/ V V Cárabos Coop. V. Enjoy Data "-"
On Thu, 6 Oct 2005, Francesc Altet wrote:
A Dijous 06 Octubre 2005 20:55, Colin J. Williams va escriure:
RESULTS:
pythonw -u "NumericNumarrayTest.py"
Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899
Is that really true? From my original post:
from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 0.12782907485961914 setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100) 1.2013700008392334
My post was from January, so, in the interim numarray has improved *a lot* it's object creation time. In that case, why the numarray team hasn't publisized that more? Well, I think I should be missing something. Time for nap.
Well, this seems pretty clear: On Thu, 14 Apr 2005, Todd Miller wrote:
Subject: [Numpy-discussion] ANN: numarray-1.3.0
I. ENHANCEMENTS
2. Removal of dictionary update from array view creation improves performance of view/slice/subarray creation. This should e.g. improve the performance of wxPython sequence protocol access to Nx2 arrays.
You might wonder why everyone didn't notice when their numarray programs started to run blazingly fast after 1.3.0. My (jaded) view is that it is very rare to have an application where the overall execution time is dominated by the time to create small arrays. When you're in that limit, there are a million other overheads that are also slowing you down. In the vast majority of applications you could reduce the array creation time to zero and probably wouldn't notice the difference without running timing tests. IMHO the numarray developers put their emphasis in the right place by focusing on large array performance and improved functionality, and all the noise around small array indexing performance was just a red herring that convinced some folks not to try numarray because they heard it was slow. I hope Travis doesn't devote a lot of effort to this optimization right now. I'd be much more interested in seeing a large array benchmark. Rick
Rick White wrote:
IMHO the numarray developers put their emphasis in the right place by focusing on large array performance and improved functionality, and all the noise around small array indexing performance was just a red herring that convinced some folks not to try numarray because they heard it was slow. I hope Travis doesn't devote a lot of effort to this optimization right now. I'd be much more interested in seeing a large array benchmark.
Except for codes like our PDE solvers, which need to create 10s of millions of small arrays very, very fast. So no, it's not a red herring [1]. Just because you don't need something doesn't mean there's no valid need for it. I am not disputing in any way the value of the many, many improvements made by numarray, nor the importance of fixing large array performance for many applications (such as I imagine is the typical workflow of astronomical data analysis). The fact that Travis tried to incorporate all of numarrays' improvements into the new library is a recognition of the value of all this work. But calling the small array performance problem a 'red herring' is inaccurate and unfair, at best. Regards, f [1] Yes, I could preallocate pools of memory for this, manage them manually, etc. But then I wouldn't be writing my code in python, would I? The whole point of using python is to have my cake and eat it too: I want high-level code that gives me near-hardware max speeds. With carefully tuned (python) code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of thinking about algorithm design, I get that.
Rick White wrote:
You might wonder why everyone didn't notice when their numarray programs started to run blazingly fast after 1.3.0.
Maybe because those of us for whom small array performance is important don't use numarray? I know I haven't tested my wxPython FloatCanvas with numarray since I discovered painfully slow performance a while back. -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov
Colin J. Williams wrote:
# NumericNumarrayTest.py ''' A test, provided by Francesc Altet
http://sourceforge.net/mailarchive/forum.php?thread_id=6396059&forum_id=4890
''' from timeit import Timer setup = "import Numeric; a = Numeric.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric23.8:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100)
# setup = "import numarray; a = numarray.arange(2000);a.shape=(1000,2)" print 'Timer-numarray1.3.3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100)
# scipy/Numeric3 added setup = "import scipy.base; a = scipy.base.arange(2000);a.shape=(1000,2)" print 'Timer-Numeric3:', Timer("for i in range(len(a)): row=a[i]", setup).timeit(number=100)
RESULTS:
pythonw -u "NumericNumarrayTest.py"
Timer-Numeric23.8: 0.179712784725 Timer-numarray1.3.3: 0.21674933546 Timer-Numeric3: 0.253077136899
After a simple optimization in the array_subscript function (to check for scalar selection up front): new results are (for 1000 runs on my system): Numeric: 0.56431102752685547 numarray: 1.0897960662841797 scipy: 0.57508182525634766 (it's now executing nearly the identical code as Numeric) Any more optimization ideas? -Travis
participants (6)
-
Chris Barker -
Colin J. Williams -
Fernando Perez -
Francesc Altet -
Rick White -
Travis Oliphant