ndarray.resize method and reference counting
Hi, I'm confused by the following:
import numpy as np np.__version__ '1.3.0.dev6116'
# I expect this
x = np.eye(3) x.resize((5,5)) x = np.eye(3) y = x x.resize((5,5)) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function
# I don't expect this
x = np.eye(3) x array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]]) x.resize((5,5), refcheck=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function x.resize((5,5), refcheck=False) x array([[ 1., 0., 0., 0., 1.], [ 0., 0., 0., 1., 0.], [ 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0.]])
Is there a reference counting bug, or am I misunderstanding something about how Python works when I type a variable's name at the prompt? Cheers, Scott
Hi Scott I can't reproduce the problem below. Would you please send a self-contained snippet? Note that, in Python, "_" is a special variable that always points to the last result. In IPython there are several others. Cheers Stéfan 2009/1/13 Scott Sinclair <scott.sinclair.za@gmail.com>:
# I don't expect this
x = np.eye(3) x array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]]) x.resize((5,5), refcheck=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function x.resize((5,5), refcheck=False)
I thought it was a self contained snippet ;-) Here's another attempt that shows "_" is the cause of my confusion.
import numpy as np x = np.eye(3) x array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]]) x.resize((5,5)) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function _ array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]])
Thanks for the help, Scott 2009/1/13 Stéfan van der Walt <stefan@sun.ac.za>:
Hi Scott
I can't reproduce the problem below. Would you please send a self-contained snippet?
Note that, in Python, "_" is a special variable that always points to the last result. In IPython there are several others.
Cheers Stéfan
2009/1/13 Scott Sinclair <scott.sinclair.za@gmail.com>:
# I don't expect this
x = np.eye(3) x array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]]) x.resize((5,5), refcheck=True) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function x.resize((5,5), refcheck=False)
Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Scott Sinclair <scott.sinclair.za <at> gmail.com> writes:
import numpy as np x = np.eye(3) x array([[ 1., 0., 0.], [ 0., 1., 0.], [ 0., 0., 1.]]) x.resize((5,5)) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function
I'm having the same problem, and have resigned to using "x = resize(x, newshape)" rather than "x.resize(newshape)" as suggested in the error message. Anything else becomes almost impossible to debug. Basically, you must never step into a function that does x.resize(). Part of the reason is explained here: http://article.gmane.org/gmane.comp.python.numeric.general/5461 In my experience, reference counting behaves differently between: * non-interactive running and simple IPython "run" of scripts (works as intended) * interactive entering of commands in IPython (entering "x" at prompt "In [n]" defines an alias _n) * debugging with "run -d" in Python (symbols in inner scopes seem to get extra references to them) See the example below (using IPython 0.8.1/Python 2.5.2/Numpy 1.1.0/Linux) for illustration. Hope this helps, Jon Olav ==== In [10]: cat test.py import numpy, sys x = numpy.zeros(3) print sys.getrefcount(x) x.resize(2) x print sys.getrefcount(x) x.resize(1) def test(): y = numpy.zeros(3) print sys.getrefcount(y) y.resize(2) y print sys.getrefcount(y) y.resize(1) test() In [11]: run test.py 2 2 2 2 ===== Non-interactively, getrefcount() returns 2 on all four occasions. Entering the same commands interactively, the refcount of x increases by 4 on entering "x" at the prompt. Inside a function call there is no problem, though. ===== In [2]: import numpy, sys In [3]: x = numpy.zeros(3) In [4]: print sys.getrefcount(x) 2 In [5]: x.resize(2) In [6]: x Out[6]: array([ 0., 0.]) In [7]: print sys.getrefcount(x) 6 In [8]: x.resize(1) --------------------------------------------------------------------------- <type 'exceptions.ValueError'>: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function In [9]: def test(): ...: y = numpy.zeros(3) ...: print sys.getrefcount(y) ...: y.resize(2) ...: y ...: print sys.getrefcount(y) ...: y.resize(1) ...: In [10]: test() 2 2 ===== When debugging, however, the x.resize() works fine, while the y.resize() inside the test() function fails. (A workaround is to step over the function call, using "n" rather than "s" in line 19.) ==== In [5]: run -d test.py ipdb> s 1---> 1 import numpy, sys ----> 3 x = numpy.zeros(3) ----> 4 print sys.getrefcount(x) 2 ----> 5 x.resize(2) ----> 6 x ----> 7 print sys.getrefcount(x) 2 ----> 8 x.resize(1) ---> 10 def test(): ---> 19 test() ipdb> s ---> 10 def test(): ---> 11 y = numpy.zeros(3) ---> 12 print sys.getrefcount(y) 3 ---> 13 y.resize(2) ValueError: 'cannot resize an array that has been referenced or is referencing \nanother array in this way. Use the resize function' =====
participants (3)
-
Jon Olav Vik -
Scott Sinclair -
Stéfan van der Walt