[AstroPy] PyFITS memory usage

[Joe Zuntz]

I've solved a similar problem in the past by importing gc and running gc.collect() after each file.

Joe Zuntz

Sent from my iPhone

On 13 May 2011, at 17:58, "G Jones" <glenn.caltech at gmail.com> wrote:

> Hello,
> I am seeing what looks like a memory leak when using PyFITS. It may not actually be a leak, but I am curious if anyone can explain the behavior. I am processing PSRFITS files which are ~40 GB and have a table with the data in it. Since the files are so large, I open them with memmap=True. I would expect the following simple script to have a constant memory footprint, but instead I see continuously growing memory usage.
> 
> import pyfits
> 
> hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True)
> primary,secondary = hdul
> for k in range(100):
>   x = secondary.data[k].field('DATA').astype('float32')
> 
> 
> I can see that the data is being read in from disk as it should be, but I don't expect the memory usage to increase, since x is constantly overwritten. The versions of programs I am using are:
> RHEL 5 x86_64 (2.6.18-194.32.1.el5)
> python 2.7.1
> numpy 2.0.0.dev
> pyfits 2.4.0
> 
> Thanks,
> Glenn
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy