In financial time series, it is very common to keep track of things like a trailing N day max, trailing N day average, etc. Generally, for a 1D array x, I'd like to be able to efficiently compute a new len(x) vector where y[i] = func(x[i-N:]) and I need to be able to handle edge effects (eg where i<N). I'm thinking of writing some extension code to handle this, but since this is a common task, I thought I'd elicit some suggestions about how to best handle this. One approach might to create a very large 2D array with each row the appropriate masked (to handle edge effects) shifted array, and then one could do sums and maxes over the columns. For a length 1000 vector with a max lag of 250, you'd have 250K elements, which is probably manageable, but I'm still not sure how to best create this shifted array and if there are better approaches. JDH
On Tuesday 30 October 2007 13:31:41 John Hunter wrote:
In financial time series, it is very common to keep track of things like a trailing N day max, trailing N day average, etc.
John, Have you ever tried the timeseries package in the scipy SVN ? We (Matt Knox and I) tried to address some of these issues for environmental and financial time series. http://www.scipy.org/SciPyPackages/TimeSeries
Pierre GM <pgmdevlist <at> gmail.com> writes:
On Tuesday 30 October 2007 13:31:41 John Hunter wrote:
In financial time series, it is very common to keep track of things like a trailing N day max, trailing N day average, etc.
John, Have you ever tried the timeseries package in the scipy SVN ? We (Matt Knox and I) tried to address some of these issues for environmental and financial time series. http://www.scipy.org/SciPyPackages/TimeSeries
I have just added a trailing max and trailing min function to the timeseries package to go along with the already existing trailing median function. These functions are implemented in C and attempt to be memory and computationally efficient, and should be much quicker than creating a 2D array as described in another post. Here is some example code...
from timeseries.lib.moving_funcs import mov_max, mov_median, mov_min import numpy as np a = np.arange(10) print mov_max(a, 4) [-- -- -- 3 4 5 6 7 8 9] print mov_median(a, 4) [-- -- -- 1.5 2.5 3.5 4.5 5.5 6.5 7.5] print mov_min(a, 4) [-- -- -- 0 1 2 3 4 5 6]
Note that the function returns a MaskedArray from the maskedarray package (currently in the sandbox), and maskedarray is a requirement for the timeseries package. You can pass TimeSeries objects, regular ndarray's, or MaskedArray's to these functions. If you decide to give the timeseries package a shot, I'm happy to help with any questions/concerns you may have. - Matt
On Tuesday 30 October 2007 13:55, Pierre GM wrote:
On Tuesday 30 October 2007 13:31:41 John Hunter wrote:
In financial time series, it is very common to keep track of things like a trailing N day max, trailing N day average, etc.
John, Have you ever tried the timeseries package in the scipy SVN ? We (Matt Knox and I) tried to address some of these issues for environmental and financial time series. http://www.scipy.org/SciPyPackages/TimeSeries
You might also want to look at ta-lib: http://ta-lib.org/ -- written by Karol Langner Tue Oct 30 20:08:13 EDT 2007
On Tue, Oct 30, 2007 at 12:31:41PM -0500, John Hunter wrote:
In financial time series, it is very common to keep track of things like a trailing N day max, trailing N day average, etc. Generally, for a 1D array x, I'd like to be able to efficiently compute a new len(x) vector where y[i] = func(x[i-N:]) and I need to be able to handle edge effects (eg where i<N). I'm thinking of writing some extension code to handle this, but since this is a common task, I thought I'd elicit some suggestions about how to best handle this. One approach might to create a very large 2D array with each row the appropriate masked (to handle edge effects) shifted array, and then one could do sums and maxes over the columns. For a length 1000 vector with a max lag of 250, you'd have 250K elements, which is probably manageable, but I'm still not sure how to best create this shifted array and if there are better approaches.
A while ago Anne posted the attached 'segmentaxis'. This is a very useful piece of code! Regards Stéfan
participants (5)
-
John Hunter -
Karol Langner -
Matt Knox -
Pierre GM -
Stefan van der Walt