[Numpy-discussion] Help making better use of numpy array functions

[Vincent Schut]

mdekauwe wrote:
> 
> Vincent Schut-2 wrote:
>> Oh, and minor issue: creating a array of zeros and then multiplying with 
>> -999 still makes an array of zeros... I'd incorporated an array of 
>> *ones* multiplied with -999, because for the last chunk of days you 
>> could end up with a 8day array only partly filled with real data. E.g. 
>> if you'd have only 3 days of data left in your last chunk, 8dayData[0:3] 
>> would be data, and to prevent the rest ([3:8]) to be incorporated into 
>> the average calculation, these need to be -999. Currently these pixels 
>> will be 0, which is > nodatavalue, and thus infuencing your average (the 
>> pixelcount will be incremented for these 0 values).
>>
> 
> Ok I hadn't thought about it in that way but you are of course right! I have
> amended it.
> 
> 
> Vincent Schut-2 wrote:
>> Alternatively, you could simply take the sum over axis=0 of the weight 
>> array to get the pixel count (e.g. "pixelcount=weight.sum(axis=0)").
>>
> 
> Ok I see your point here as well. So I tried implementing your suggestion,
> as I understand it
> 
> weights = data8days > nodatavalue
> 
> will make and 8, nrows, ncols array containing true and false.
> 
> as you said I can get the pixel count I was after by using
> weights.sum(axis=0).
> 
> However when I try to do the averaging step:
> 
> avg8days = np.average(data8days, axis=0, weights=weights)
> 
> I get the error msg " in average raise ZeroDivisionError, "Weights sum to
> zero, can't be normalized"
> ZeroDivisionError: Weights sum to zero, can't be normalized"
> 
> Which I guess (but don't know) comes from the trying do weight by a pixel
> count of zero. So not sure what has happened here?
> 
> Thanks 
> 

Ah... apparently numpy.average can't handle the situation when all 
weights are 0... didn't know that.
Hmm... what would you want to have in your average array when for a 
certain pixel there are only nodata values? If you'd like to have -999 
in your average result then, the solution is simple: in the weight 
array, for those pixels where weight is always 0, set 1 dayweight to 1. 
this way you'll get a nice -999 in your average result for those pixels.
e.g.:

weights[0] |= (weights.sum(axis=0)==0)

will set (boolean OR assign) all pixels with weight==0 for all days to 
True (==1).

Hope this helps.
Vincent.