On 31/10/2007, Ray S <subscriber100@rjs.org> wrote:

I am using fftRes = abs(fft.rfft(data_array[end-2**15:end])) to do running analysis on streaming data. The N never changes. It sucks memory up at ~1MB/sec with 70kHz data rate and 290 ffts/sec. (Interestingly, Numeric FFT accumulates much slower..) (Commenting out that one line stops memory growth.)

What can one do to alleviate this? Can I del() some numpy object or such? It's a bit of an issue for a program that needs to run for weeks.

It's purpose is simply to argmax() the largest bin, which always falls within a small range - do I have another, better option than fft?

If the range is *really* small, you can try using a DFT - sometimes that is fast enough, and gives you just the bins you're curious about. If the range is bigger than that, but still a small fraction of the FFT size, you can do some tricks where you band-pass filter the data (with a FIR filter, say), downsample (which aliases frequencies downward), and then FFT. Concretely, if your data is sampled at 80ksamp/s and you're taking 160ksamp FFTs, you're getting the spectrum up to 35 kHz with a resolution of 0.5 Hz. If all you care about is the spectral region from 5 kHz to 6 kHz but you still need the 0.5 Hz spectral resolution, you can run a band-pass filter to throw out everything outside (say) 4 to 7 kHz, then downsample to 8 ksamp/s; the 4 to 7kHz region will appear mirrored in the 1 to 4 kHz region of the new FFTs, which will each be a tenth the size. There are also "zoom fft" and "chirp-z" techniques which are supposed to give you only part of the FFT, but the wisdom is that unless you want less than a few percent of the data points you're better just FFTing and throwing lots of data away. Anne