[Numpy-discussion] Re: Garbage collection fails after numarray exception

[Todd Miller]

On Sun, 2004-02-29 at 04:28, Leo Breebaart wrote:
> Rob Hooft <rob at hooft.net> writes:
> 
> > > On Tue, 2004-02-10 at 15:09, Leo Breebaart wrote:
> > > 
> > >>   >>> import numarray
> > >>   >>> a = numarray.zeros(100000000)
> > >>   >>> del a
> > >>   >>> a = numarray.zeros(100000000)
> > >>   >>> a + ''
> > >>   Traceback (most recent call last):
> > >>     File "<stdin>", line 1, in ?
> > >>     File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__
> > >>       return ufunc.add(self, operand)
> > >>     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2
> > >>       (in1, in2), insig, scalar = _inputcheck(n1, n2)
> > >>     File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck    raise TypeError(
> > >>   TypeError: UFunc arguments must be numarray, scalars or numeric sequences
> > >>   >>> del a
> > >>   >>>
> > >>
> > >> If I execute these statements alongside 'top' or another
> > >> memory monitor, I of course see a big memory increase after
> > >> each call to 'zeros', as well as a big decrease after the
> > >> first 'del a' -- but no change whatsoever any more after the
> > >> second 'del a', not even if I explicitly call the garbage
> > >> collector via the gc module. [...]
> > 
> > Hmm.... Isn't this the standard python thing that it keeps the
> > context of the last exception somewhere for queries? Just

This is a good point, something I rediscovered for myself a couple weeks
ago:  since Python holds onto the context at the point of an exception, 
it will keep arrays alive in the exception context.

> > generate another exception and you should see that the memory
> > taken by a is liberated because the indirect reference from
> > sys.exc_traceback to a disappears.
> 
> Well, I've tried to test this, but even after multiple subsequent
> exceptions the memory taken up by the numarrays is not freed, at
> least not on my machine and with Python 2.3.3. Does it work for
> you if you try running the above code and then generating an
> exception?

But... I looked into this some more and found that there were errors in
the exception handling for ufuncs which resulted in leaked memory.

Regards,
Todd

-- 
Todd Miller <jmiller at stsci.edu>