Re: [Numpy-discussion] numpy FFT memory accumulation
At 09:00 AM 11/1/2007, you wrote: I saw that Numeric did also (I still use Numeric for smaller array speed) but much more slowly. I will try to repeat with a small demo and post. It turns out to be some aspect of mixing numpy and Numeric; the attached *Stable.py files allocate memory that stays exactly the same mixedGrows.py grows by about 50kB/s while changing slightly, randomly; note that the array is assigned as numpy and operated on with Numeric functions. My main app also uses mmap/numpy to assign the arrays to shared memory for multiple processes, and does other operations as well, possibly accounting for the faster memory growth. I'll re-factor to an all-numpy solution and test. I used Numeric functions for the ~40% speed increase, but I don't have a compiled version of arrayfrombuffer() for Numeric http://www.canonical.org/~kragen/sw/arrayfrombuffer/ (compiler version woes) so, I must use numpy.frombuffer() with mmap to create the shared arrays... Ray
On 11/1/07, Ray S
At 09:00 AM 11/1/2007, you wrote: I saw that Numeric did also (I still use Numeric for smaller array speed) but much more slowly. I will try to repeat with a small demo and post.
It turns out to be some aspect of mixing numpy and Numeric;
the attached *Stable.py files allocate memory that stays exactly the same
mixedGrows.py grows by about 50kB/s while changing slightly, randomly; note that the array is assigned as numpy and operated on with Numeric functions.
My main app also uses mmap/numpy to assign the arrays to shared memory for multiple processes, and does other operations as well, possibly accounting for the faster memory growth. I'll re-factor to an all-numpy solution and test.
I used Numeric functions for the ~40% speed increase, but I don't
I know that numarray was slow in creating small arrays, but is Numpy really that bad compared to Numeric? Chuck
participants (2)
-
Charles R Harris
-
Ray S