[AstroPy] pyfits - memory leak in new_table / 'hello, i am new here'
embray at stsci.edu
Wed Jun 15 11:22:55 EDT 2011
On 06/15/2011 08:38 AM, Martin Raue wrote:
> Hi Stefan,
>> I tried your script with revision 851 and the memory leak is there.
>> Then I tried with trunk (currently 925) and there is no more memory leak.
> Thanks for checking!
>> The problem you had with the installation happened to me as well, but it
>> worked for me with:
>> sudo ./setup.py install
> Unfortunately, that produces the same error (also r925). Erik Bray wrote me that he will have a look at the installer.
> In your case, it could be that your user and your superuser have different python version (and/or setups) installed or that /usr/bin/env points toward a different version of python then the python command.
> Cheers, m
Yes, the problem here is with a semi-experimental tool called d2to1 that
I developed and am using in the packaging of pyfits.
(For those interested, its purpose is to make distutils2/packaging-style
setup.cfg files work with setuptools/distribute).
That particular bug can only happen if you didn't already have d2to1
installed. I should have a fix for it checked in shortly.
As for the actual memory issue: I confirmed that on trunk there is no
memory leak. But it is worth pointing out that pyfits.new_table() uses
more memory than one might expect. This is because it allocates a new
array that has space for all the columns that will be added to it. The
original column arrays can't be use directly, because they have to be
But if you're having trouble with memory usage when building tables,
there are ways of dealing with this. I can make specific suggestions if
you're looking for ideas on how to go about it.
More information about the AstroPy