Hi, I've got two issues: First, the following seems to cause a memory leak, using numpy 1.3.0: a = matrix(ones(1)) while True: a += 0 This only seems to happen when a is a matrix rather than an array, and when the short hand '+=' is used. Second, I'm not sure whether that's a bug or whether I just don't understand what's going on, but when a is a column array, (e.g. a = ones((10, 1))), then a -= a[0,:] only subtracts from a[0, 0], whereas not using the short hand or using something else than a on the righthand side seems to subtract from all rows as expected. Thanks a lot, David -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
On Tue, Mar 9, 2010 at 12:31 PM, David Paul Reichert <D.P.Reichert@sms.ed.ac.uk> wrote:
Hi,
I've got two issues:
First, the following seems to cause a memory leak, using numpy 1.3.0:
a = matrix(ones(1))
while True: a += 0
This only seems to happen when a is a matrix rather than an array, and when the short hand '+=' is used.
Second, I'm not sure whether that's a bug or whether I just don't understand what's going on, but when a is a column array, (e.g. a = ones((10, 1))), then
a -= a[0,:]
only subtracts from a[0, 0], whereas not using the short hand or using something else than a on the righthand side seems to subtract from all rows as expected.
this is because a[0,0] is set to zero after the first inplace subtraction, then zero is subtracted from all other rows
a = np.ones((10, 1)) a array([[ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.]]) a += a[0,:] a array([[ 2.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.]]) a -= a[0,:] a array([[ 0.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.]])
Josef
Thanks a lot,
David
-- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Thanks for the reply. Yes never mind the second issue, I had myself confused there. Any comments on the memory leak? On Tue, Mar 9, 2010 at 5:55 PM, <josef.pktd@gmail.com> wrote:
On Tue, Mar 9, 2010 at 12:31 PM, David Paul Reichert <D.P.Reichert@sms.ed.ac.uk> wrote:
Hi,
I've got two issues:
First, the following seems to cause a memory leak, using numpy 1.3.0:
a = matrix(ones(1))
while True: a += 0
This only seems to happen when a is a matrix rather than an array, and when the short hand '+=' is used.
Second, I'm not sure whether that's a bug or whether I just don't understand what's going on, but when a is a column array, (e.g. a = ones((10, 1))), then
a -= a[0,:]
only subtracts from a[0, 0], whereas not using the short hand or using something else than a on the righthand side seems to subtract from all rows as expected.
this is because a[0,0] is set to zero after the first inplace subtraction, then zero is subtracted from all other rows
a = np.ones((10, 1)) a array([[ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.], [ 1.]]) a += a[0,:] a array([[ 2.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.]]) a -= a[0,:] a array([[ 0.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.], [ 3.]])
Josef
Thanks a lot,
David
-- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.
On Tue, Mar 9, 2010 at 11:31, David Paul Reichert <D.P.Reichert@sms.ed.ac.uk> wrote:
Hi,
I've got two issues:
First, the following seems to cause a memory leak, using numpy 1.3.0:
a = matrix(ones(1))
while True: a += 0
This only seems to happen when a is a matrix rather than an array, and when the short hand '+=' is used.
Yes, I can verify that there is a leak in the current SVN, too. It appears that we are creating dictionaries with the value {'_getitem': False} and never decrefing them. The matrix object itself owns the reference.
Second, I'm not sure whether that's a bug or whether I just don't understand what's going on, but when a is a column array, (e.g. a = ones((10, 1))), then
a -= a[0,:]
only subtracts from a[0, 0], whereas not using the short hand or using something else than a on the righthand side seems to subtract from all rows as expected.
a[0,:] creates a view onto the same memory of the original array. Since you modify the values in-place, a[0,0] gets set to a[0,0]-a[0,0]==0, then a[1,0] gets set to a[1,0] - a[0,0] == a[1,0] - 0 == a[1,0], etc. Try this instead: a -= a[0,:].copy() -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Tue, Mar 9, 2010 at 1:28 PM, Robert Kern <robert.kern@gmail.com> wrote:
On Tue, Mar 9, 2010 at 11:31, David Paul Reichert <D.P.Reichert@sms.ed.ac.uk> wrote:
Hi,
I've got two issues:
First, the following seems to cause a memory leak, using numpy 1.3.0:
a = matrix(ones(1))
while True: a += 0
This only seems to happen when a is a matrix rather than an array, and when the short hand '+=' is used.
Yes, I can verify that there is a leak in the current SVN, too. It appears that we are creating dictionaries with the value {'_getitem': False} and never decrefing them. The matrix object itself owns the reference.
With numpy 1.4.0 memory usage also keeps growing until I kill the process. Josef
Second, I'm not sure whether that's a bug or whether I just don't understand what's going on, but when a is a column array, (e.g. a = ones((10, 1))), then
a -= a[0,:]
only subtracts from a[0, 0], whereas not using the short hand or using something else than a on the righthand side seems to subtract from all rows as expected.
a[0,:] creates a view onto the same memory of the original array. Since you modify the values in-place, a[0,0] gets set to a[0,0]-a[0,0]==0, then a[1,0] gets set to a[1,0] - a[0,0] == a[1,0] - 0 == a[1,0], etc. Try this instead:
a -= a[0,:].copy()
-- Robert Kern
"I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
participants (4)
-
David Paul Reichert -
David Reichert
-
josef.pktd＠gmail.com -
Robert Kern