[Numpy-discussion] custom accumlators

[Matt Knox]

I made a post about this a while ago on the scipy-user mailing list, but I didn't receive much of a response so I'm just throwing it out there again (with more detail) in case it got overlooked.

Basically, I'd like to be able to do accumulate operations with custom functions. numpy.vectorize does not seem to provide an accumulate method with the functions it returns. I'm hoping I don't have to write ufuncs in C to accomplish this, but I fear that may the case. Either way, it would be nice to know if it can, or cannot be done in an easy manner.

I have lots of examples of where this kind of thing is useful, but I'll just outline two for now.

Assume the parameter x in all the functions below is a 1-d array

---------------------------------------------
Example 1 - exponential moving average:

# naive brute force method...
def expmave(x, k):
    result = numpy.array(x, copy=True)
    for i in range(1, result.size):
       result[i] = result[i-1] + k * (result[i] - result[i-1])
    return result

# slicker method (if it worked, which it doesn't)...
def expmave(x, k):
    def expmave_sub(a, b):
        return a + k * (b - a)
    return numpy.vectorize(expmave_sub).accumulate(x)
    
---------------------------------------------
Example 2 - forward fill a masked array:

# naive brute force method...
def forward_fill(x):
    result = ma.array(x, copy=True)
    for i in range(1, result.size):
       if result[i] is ma.masked: result[i] = result[i-1]
    return result

# slicker method (if it worked, which it doesn't)...
def forward_fill(x):
    def forward_fill_sub(a, b):
    	if b is ma.masked: return a
    	else: return b
    return numpy.vectorize(forward_fill_sub).accumulate(x)
---------------------------------------------

Is their a good way to do these kinds of things without python looping? Or is that going to require writing a ufunc in C? Any help is greatly appreciated.

Thanks,

- Matt Knox