[Numpy-discussion] RE: [Numpy-developers] [Bug #128025] MA seems to leak memory

[Paul F. Dubois]

The helpful person who submitted this report asks for a reply but does not
give their name. Thank you for finding this problem.

It turns out that this is and this isn't a problem. On the whole, it is. I
have a fix but need to warn people that it will break any existing code that
takes advantage of a feature I did not advertise. (:->

Here's the story: class MA inherits from a class ActiveAttributes, supplied
with MA as an announced module in the package. ActiveAttributes is designed
to encapsulate the behavior that Numeric has, of having attributes that are
not really attributes but are trios of functions. For an example of what I
mean by this, consider the .shape attribute.
print x.shape
x.shape = (3, 9)
del x.shape

shape appears to be the name of an attribute but in fact it is not. There
are actually a trio of handlers, so that the above actually execute more
like they were
print x.__getshape()
x.__setshape((3,9))
x.__delattr('shape')

Now for the problem. In implementing ActiveAttributes, I set up an indirect
system of handlers for each "active" attribute like 'shape', and in
remembering what handlers to use I carelessly used *bound* methods rather
than *unbound* methods. This meant that each instance contained a reference
cycle. However, Python 2.0 has a cyclic garbage collector that runs
periodically so over the course of a long routine like my test routine, the
garbage was in fact being collected. The bug-poster's patch reveals the
problem by doing it a lot of memory operations in a very few statements.

The fix is to change ActiveAttributes to save unbound rather than bound
methods. This works and prevents the observed growth. I will check it in to
CVS with a new release number 4.0 when I have finished testing another idea
that may also be an improvement. Current users may wish to invoke the
facilities of the "gc" module in 2.0 to increase the frequency of collection
if they are currently experiencing a problem. To anyone out there who had
noticed the activeattr.py module and used it, this change will require an
incompatible change to your code when you switch to the new version.

-----Original Message-----
From: numpy-developers-admin at lists.sourceforge.net
[mailto:numpy-developers-admin at lists.sourceforge.net]On Behalf Of
noreply at sourceforge.net
Sent: Monday, January 08, 2001 4:27 AM
To: noreply at sourceforge.net; noreply at sourceforge.net;
numpy-developers at sourceforge.net
Subject: [Numpy-developers] [Bug #128025] MA seems to leak memory


Bug #128025, was updated on 2001-Jan-08 04:26
Here is a current snapshot of the bug.

Project: Numerical Python
Category: Fatal Error
Status: Open
Resolution: None
Bug Group: Robustness lacking
Priority: 5
Submitted by: nobody
Assigned to : nobody
Summary: MA seems to leak memory

Details: I executed the following code in the interpreter:
>>> import MA
>>> a = MA.arange(10000000)
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
>>> b = a[:5000000]
After array a was created, memory consumption was around 40 MB.
After the b slices, it's now 190 MB.
Even if MA does make a copy-on-slice, it should IMHO free the slice after
there are no more references to it.

(I am not a Sourceforge user or anything, so I would appreciate if someone
would at least let me know if this is a known problem.)

For detailed info, follow this link:
http://sourceforge.net/bugs/?func=detailbug&bug_id=128025&group_id=1369

_______________________________________________
Numpy-developers mailing list
Numpy-developers at lists.sourceforge.net
http://lists.sourceforge.net/mailman/listinfo/numpy-developers