[Numpy-discussion] summing "sparse" 2D arrays? Results...

Ray Schumacher rays at blue-cove.com
Mon Jan 26 11:55:09 EST 2004


I just realized...
where() belongs to Numeric, so I need
sum([Numeric.where(a < 255, a, 0) for a in y])
duh.

I did just compare Numeric vs. Masked arrays:
===========================================================
# test.py
from MA import masked_array, sum
from RandomArray import *
import time

seed()
y = randint(240,256, (480,640,16))
start = time.time()

x=masked_array(y, y>=255)
maskTime = time.time() - start
sum_1 = sum(x,axis=2)
maskedTime = time.time() - start
print sum_1.shape
print sum_1
print "mask make time: " + str(maskTime)
print "time using MA: " + str(maskedTime) + "\n"

z = Numeric.reshape(y, (16, 480, 640))
newStart = time.time()
sum_2 = sum([Numeric.where(a < 255, a, 2) for a in z])
numTime = time.time() - newStart
print sum_2.shape
print sum_2
print "time using Numeric: " + str(numTime) + "\n"
======================================================

Result:
C:\projects\Astro>python test.py
(480, 640)
array (480,640) , type = O, has 307200 elements
mask make time: 1.07899999619
time using MA: 3.39100003242

(480, 640)
array (480,640) , type = l, has 307200 elements
time using Numeric: 2.39099979401

So, MA's sum() is slightly faster, but the penalty for making a mask first 
is large.

Now I have to figure out why I had to reshape the array for the second 
computation.

Thanks,
Ray












At 09:17 AM 1/26/2004 +0100, you wrote:
>On 26.01.2004, at 07:14, RJS wrote:
>
>>The problem: I have a "stack" of 8,  640 x 480 integer image arrays from 
>>a FITS cube concatenated into a 3D array, and I want to sum each pixel 
>>such that the result ignores clipped values (255+); i.e., if two images 
>>have clipped pixels at (x,y) the result along z will be the sum of the other 6.
>Memory doesn't seem critical for such small arrays, so you can just do
>
>sum([where(a < 255, a, 0) for a in images])

Hello Konrad,

I just tried:

from MA import masked_array, sum
from RandomArray import *
seed()
y = randint(240,256, (480,640,2))
print sum([where(a < 255, a, 0) for a in y])

and it errors:
Traceback (most recent call last):
   File "test.py", line 21, in ?
     print sum([where(a < 255, a, 0) for a in y])
NameError: name 'where' is not defined

Could you enlighten me further?
I have not found a good resource for compound Numeric statements yet.

Thank you,

Ray






More information about the NumPy-Discussion mailing list