Downsampling a 2D array with min/max and nullvalues
Hi there, I have a 2D array of size, lets say 40000 * 512, which I need to downsample by a step of 4 in the y direction, and step 3 in the x direction, so I would get an array of 10000 * 170 (or 171, the edges do not matter much). But what I need to retain are the maxima and minima in every 4*3 window. I have a solution that works by first clipping off the edges of the array, so that its shape is divisible by 4*3, and then applying a two step process that will compute the min and max in strips --first in the x direction and then in the y-direction through reshaping and reduce (not super elegant, but it seems to work): def downsampleArray1D(data, step, nullValue): print 'downsampling %d' %step if data.shape[1] % step != 0: xlen = data.shape[1] - data.shape[1] % step data = data[...,:xlen] data1d = data.ravel() assert(data1d.shape[0] % step == 0) data1d.shape = (len(data1d)/step, step) maxdata = numpy.maximum.reduce(data1d,1) mindata = numpy.minimum.reduce(data1d,1) mindata.shape = ((data.shape[0],int(data.shape[1] / step))) maxdata.shape = ((data.shape[0],int(data.shape[1] / step))) return mindata, maxdata def downsampleArray2D(data, yStep, xStep, nullValue): # first adjust the data to the step size by ignoring edges if data.shape[0] % yStep != 0: ylen = data.shape[0] - data.shape[0] % yStep data = data[:ylen,...] if data.shape[1] % xStep != 0: xlen = data.shape[1] - data.shape[1] % xStep data = data[...,:xlen] minx, maxx = downsampleArray1D(data, xStep, nullValue) minxt = numpy.transpose(minx) minxt1, dummy = downsampleArray1D(minxt, yStep, nullValue) maxxt = numpy.transpose(maxx) dummy, maxxt1 = downsampleArray1D(maxxt, yStep, nullValue) minimum = numpy.transpose(minxt1) maximum = numpy.transpose(maxxt1) return minimum, maximum Now I need a solution that will ignore null values, i.e. that any value that is equivalent to e.g. -999 ignored in the min and max computation, but if all values in a 4*3 window are nullvalues, then min and max should be set to the null value. Any suggestions? Ludwig
Ludwig, Masked arrays will do exactly what you want. You have your choice of the numpy.ma version or the external maskedarray class. Eric Ludwig M Brinckmann wrote:
Hi there,
I have a 2D array of size, lets say 40000 * 512, which I need to downsample by a step of 4 in the y direction, and step 3 in the x direction, so I would get an array of 10000 * 170 (or 171, the edges do not matter much). But what I need to retain are the maxima and minima in every 4*3 window.
I have a solution that works by first clipping off the edges of the array, so that its shape is divisible by 4*3, and then applying a two step process that will compute the min and max in strips --first in the x direction and then in the y-direction through reshaping and reduce (not super elegant, but it seems to work):
def downsampleArray1D(data, step, nullValue): print 'downsampling %d' %step if data.shape[1] % step != 0: xlen = data.shape[1] - data.shape[1] % step data = data[...,:xlen] data1d = data.ravel() assert(data1d.shape[0] % step == 0) data1d.shape = (len(data1d)/step, step) maxdata = numpy.maximum.reduce(data1d,1) mindata = numpy.minimum.reduce(data1d,1) mindata.shape = ((data.shape[0],int(data.shape[1] / step))) maxdata.shape = ((data.shape[0],int(data.shape[1] / step))) return mindata, maxdata
def downsampleArray2D(data, yStep, xStep, nullValue): # first adjust the data to the step size by ignoring edges if data.shape[0] % yStep != 0: ylen = data.shape[0] - data.shape[0] % yStep data = data[:ylen,...] if data.shape[1] % xStep != 0: xlen = data.shape[1] - data.shape[1] % xStep data = data[...,:xlen] minx, maxx = downsampleArray1D(data, xStep, nullValue) minxt = numpy.transpose(minx) minxt1, dummy = downsampleArray1D(minxt, yStep, nullValue)
maxxt = numpy.transpose(maxx) dummy, maxxt1 = downsampleArray1D(maxxt, yStep, nullValue)
minimum = numpy.transpose(minxt1) maximum = numpy.transpose(maxxt1)
return minimum, maximum
Now I need a solution that will ignore null values, i.e. that any value that is equivalent to e.g. -999 ignored in the min and max computation, but if all values in a 4*3 window are nullvalues, then min and max should be set to the null value.
Any suggestions?
Ludwig
------------------------------------------------------------------------
_______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
participants (2)
-
Eric Firing -
Ludwig M Brinckmann