[AstroPy] PyFITS memory usage

[G Jones]

Hello,
I am seeing what looks like a memory leak when using PyFITS. It may not
actually be a leak, but I am curious if anyone can explain the behavior. I
am processing PSRFITS files which are ~40 GB and have a table with the data
in it. Since the files are so large, I open them with memmap=True. I would
expect the following simple script to have a constant memory footprint, but
instead I see continuously growing memory usage.

import pyfits

hdul = pyfits.open('myfile.fits',mode="readonly", memmap=True)
primary,secondary = hdul
for k in range(100):
  x = secondary.data[k].field('DATA').astype('float32')


I can see that the data is being read in from disk as it should be, but I
don't expect the memory usage to increase, since x is constantly
overwritten. The versions of programs I am using are:
RHEL 5 x86_64 (2.6.18-194.32.1.el5)
python 2.7.1
numpy 2.0.0.dev
pyfits 2.4.0

Thanks,
Glenn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110513/bf5c64f2/attachment.html>