[AstroPy] pyfits - memory leak in new_table / 'hello, i am new here'

[Stefan Schwarzburg]

Hi Martin,

I tried your script with revision 851 and the memory leak is there.
Then I tried with trunk (currently 925) and there is no more memory leak.

The problem you had with the installation happened to me as well, but it
worked for me with:

sudo ./setup.py install

instead of

sudo python setup.py install


I have not checked why, but anyway it works. So maybe you just try that.

Cheers,
Stefan

On Tue, Jun 14, 2011 at 12:45, Martin Raue <martin.raue at desy.de> wrote:

> Hello all,
>
> I am new here. I work in very high energy gamma-ray astronomy with HESS and
> CTA and currently developing some high level analysis tools for CTA using
> python (the classical numpy, scipy, matplotlib, pyfits).
>
> I stumbled upon a problem with memory leakage in pyfits when using
> new_table.
>
> Something similar has been reported here
>
> http://physicsnlinux.wordpress.com/2011/03/28/pyfits-memory-leak-in-new_table/#more-103
> and should be fixed in
> http://trac6.assembla.com/pyfits/changeset/844
>
> But I still have problems in 844 and 851. Unfortunately, I have problems
> installing the latest trunk on my system (some output attached).
>
> Any suggestion and help would be very welcome!
>
> Best wishes,
> Martin
>
> ==============
> Example script
>
> import numpy as np
> import pyfits
> import gc
>
> for i in range(20) :
>
>    # create dummy data
>    a = np.linspace(-1., 1., 1000000)
>    b = a ** 3.
>
>    a_col = pyfits.Column(name='ACOL', format='1E', array=a)
>    b_col = pyfits.Column(name='BCOL', format='1E', array=b)
>
>    coldefs_new = pyfits.ColDefs([a_col, b_col])
>
>    newtable = pyfits.new_table(coldefs_new)
>
>    # try to clean memory
>    del newtable
>    del coldefs_new
>    gc.collect()
>
> gc.collect()
>
> =======================================================
> setup.py output when trying to install the pyfits trunk
>
> Something wrong in the installer modules ...
>
> [uh2macastro04:~/tmp/trunk] $ python setup.py install
> Downloading
> http://pypi.python.org/packages/source/d/distribute/distribute-0.6.15.tar.gz
> Extracting in /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s
> Now working in
> /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/tmp4nv73s/distribute-0.6.15
> Building a Distribute egg in /Users/mraue/tmp/trunk
> /Users/mraue/tmp/trunk/distribute-0.6.15-py2.6.egg
> zip_safe flag not set; analyzing archive contents...
>
> Installed
> /private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg
> zip_safe flag not set; analyzing archive contents...
> stsci.__init__: module references __path__
>
> Installed /Users/mraue/tmp/trunk/stsci.distutils-0.2dev_r13055-py2.6.egg
> Searching for d2to1>=0.1.5
> Reading http://pypi.python.org/simple/d2to1/
> Best match: d2to1 0.1.5
> Downloading
> http://pypi.python.org/packages/source/d/d2to1/d2to1-0.1.5.tar.gz#md5=34129f809b503449b283771b5c3f038b
> Processing d2to1-0.1.5.tar.gz
> Running d2to1-0.1.5/setup.py -q bdist_egg --dist-dir
> /var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-ABs3GN/d2to1-0.1.5/egg-dist-tmp-oSIXxB
> Traceback (most recent call last):
>  File "setup.py", line 33, in <module>
>    use_2to3=True
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py",
> line 113, in setup
>    _setup_distribution = dist = klass(attrs)
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 221, in
> __init__
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 245, in
> fetch_build_eggs
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 544, in
> resolve
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 786, in
> best_match
>  File "build/bdist.macosx-10.3-fat/egg/pkg_resources.py", line 798, in
> obtain
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/dist.py", line 293, in
> fetch_build_egg
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 584, in easy_install
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 614, in install_item
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 804, in install_eggs
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 1081, in build_and_install
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/command/easy_install.py",
> line 1070, in run_setup
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 29, in
> run_setup
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 70, in
> run
>  File "build/bdist.macosx-10.3-fat/egg/setuptools/sandbox.py", line 31, in
> <lambda>
>  File "setup.py", line 21, in <module>
>    except ImportError:
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py",
> line 152, in setup
>    dist.run_commands()
>  File
> "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py",
> line 975, in run_commands
>    self.run_command(cmd)
>  File
> "/private/var/folders/NT/NTbrVxJ-Gw4ybwwyECKVcE+++TM/-Tmp-/easy_install-NTCEXF/stsci.distutils-0.2dev-r13055/d2to1-0.1.5-py2.6.egg/d2to1/core.py",
> line 111, in run_command
> AttributeError: 'NoneType' object has no attribute 'info'
>
> [uh2macastro04:~/tmp/trunk] $ python
> Python 2.6.5 (r265:79359, Mar 24 2010, 01:32:55)
> [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
>
> -----------------------------------
>
> Martin Raue
>
> University of Hamburg
> Institute for Experimental Physics
> Luruper Chaussee 149
> D-22761 Hamburg
> Germany
>
> Phone  : +49-40-8998-5216
> Fax    : +49-40-8998-2170
> E-Mail : martin.raue at desy.de
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at scipy.org
> http://mail.scipy.org/mailman/listinfo/astropy
>



-- 
Institut für Astronomie und Astrophysik
Eberhard Karls Universität Tübingen
Sand 1   -  D-72076 Tübingen
Tel.: 07071/29-78605
-----------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20110615/62f20cdf/attachment.html>