masked_array/matplotlib issue with memmaps

If I initialize an AxesImage using a np.zeros array and then set the axes data later to a np.memmap array, I get a RuntimeError when matplotlib tries to autoscale the image. The errors continue to fill my console and I'm forced to close the shell. This bug was introduced when I switched from numpy v1.0.3.1 to the trunk v1.0.5.dev4815 The two hacks to get around this are: 1) Setting any array element to something other than zero fixes the error: zdata[0,0] = 1 2) Specify the extent and max/min values when creating the image: imgaxes = pylab.imshow(zdata, extent=(0, data_shape[1], data_shape[0], 0), vmin=0, vmax=1) Unfortunately, due to the way this errors I'm having a difficult time debugging it. I'm hoping someone with in-depth knowledge of masked_arrays will have some insight. Code and output are below. Thanks! Chris ---- script to reproduce the bug ---- import pylab import numpy as np def printinfo(imgaxes): a = imgaxes.get_array() print '\nimgaxes array info:' print 'type', type(a) print 'shape', a.shape print 'dtype', a.dtype print 'has _mmap', hasattr(a, '_mmap') data_type = 'float32' data_shape = (30, 40) zdata = np.zeros(data_shape, dtype=data_type) #zdata[0,0] = 1 # No exception raised if this line is executed imgaxes = pylab.imshow(zdata) printinfo(imgaxes) mmdata = np.memmap('foo.dat', dtype=zdata.dtype, shape=zdata.shape, mode='w+') imgaxes.set_data(mmdata) printinfo(imgaxes) # imgaxes array now has a _mmap pylab.show() ---- version info ---- In [2]: pylab.matplotlib.__version__ Out[2]: '0.91.2' In [4]: numpy.version.version Out[4]: '1.0.5.dev4817' ---- error ---- In [26]: run memmap_reassign.py imgaxes array info: type <class 'numpy.ma.core.MaskedArray'> shape (30, 40) dtype float32 has _mmap False imgaxes array info: type <class 'numpy.ma.core.MaskedArray'> shape (30, 40) dtype float32 has _mmap True Exception exceptions.RuntimeError: 'maximum recursion depth exceeded' in ignored ERROR: An unexpected error occurred while tokenizing input The following traceback may be corrupted or invalid The error message is: ('EOF in multi-line statement', (10, 0)) --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_wx.pyc in _onPaint(self, evt) 1079 self.realize() 1080 # Render to the bitmap -> 1081 self.draw(repaint=False) 1082 # Update the display using a PaintDC 1083 self.gui_repaint(drawDC=wx.PaintDC(self)) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_wxagg.pyc in draw(self, repaint) 59 """ 60 DEBUG_MSG("draw()", 1, self) ---> 61 FigureCanvasAgg.draw(self) 62 63 self.bitmap = _convert_agg_to_wx_bitmap(self.get_renderer(), None) /Users/cburns/local/lib/python2.5/site-packages/matplotlib/backends/backend_agg.pyc in draw(self) 356 357 self.renderer = self.get_renderer() --> 358 self.figure.draw(self.renderer) 359 360 def get_renderer(self): /Users/cburns/local/lib/python2.5/site-packages/matplotlib/figure.pyc in draw(self, renderer) 622 623 # render the axes --> 624 for a in self.axes: a.draw(renderer) 625 626 # render the figure text /Users/cburns/local/lib/python2.5/site-packages/matplotlib/axes.pyc in draw(self, renderer, inframe) 1303 mag = renderer.get_image_magnification() 1304 ims = [(im.make_image(mag),0,0) -> 1305 for im in self.images if im.get_visible()] 1306 1307 /Users/cburns/local/lib/python2.5/site-packages/matplotlib/image.pyc in make_image(self, magnification) 129 im.is_grayscale = False 130 else: --> 131 x = self.to_rgba(self._A, self._alpha) 132 im = _image.fromarray(x, 0) 133 if len(self._A.shape) == 2: /Users/cburns/local/lib/python2.5/site-packages/matplotlib/cm.pyc in to_rgba(self, x, alpha, bytes) 74 x = ma.asarray(x) 75 x = self.norm(x) ---> 76 x = self.cmap(x, alpha=alpha, bytes=bytes) 77 return x 78 /Users/cburns/local/lib/python2.5/site-packages/matplotlib/colors.pyc in __call__(self, X, alpha, bytes) 431 vtype = 'array' 432 xma = ma.asarray(X) --> 433 xa = xma.filled(0) 434 mask_bad = ma.getmask(xma) 435 if xa.dtype.char in npy.typecodes['Float']: /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in filled(self, fill_value) 1542 m = self._mask 1543 if m is nomask or not m.any(): -> 1544 return self._data 1545 # 1546 if fill_value is None: /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in __str__(self) 1614 m = self._mask 1615 if m is nomask: -> 1616 res = self._data 1617 else: 1618 if m.shape == (): /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in __str__(self) 1614 m = self._mask 1615 if m is nomask: -> 1616 res = self._data 1617 else: 1618 if m.shape == (): /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) /Users/cburns/local/lib/python2.5/site-packages/numpy/core/memmap.pyc in __array_finalize__(self, obj) 204 self._mmap = obj._mmap 205 else: --> 206 raise ValueError, 'Cannot create a memmap from object %s'%obj 207 else: 208 self._mmap = None .... [snip] /Users/cburns/local/lib/python2.5/site-packages/numpy/ma/core.pyc in _get_data(self) 1472 1473 """ -> 1474 return self.view(self._baseclass) 1475 _data = property(fget=_get_data) 1476 data = property(fget=_get_data) RuntimeError: maximum recursion depth exceeded Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in ignored Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in ignored [snip] Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])> ignored Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])> ignored [snip]

On Tue, Feb 26, 2008 at 5:26 PM, Christopher Burns <cburns@berkeley.edu> wrote:
If I initialize an AxesImage using a np.zeros array and then set the axes data later to a np.memmap array, I get a RuntimeError when matplotlib tries to autoscale the image. The errors continue to fill my console and I'm forced to close the shell. This bug was introduced when I switched from numpy v1.0.3.1 to the trunk v1.0.5.dev4815
The two hacks to get around this are: 1) Setting any array element to something other than zero fixes the error: zdata[0,0] = 1 2) Specify the extent and max/min values when creating the image: imgaxes = pylab.imshow(zdata, extent=(0, data_shape[1], data_shape[0], 0), vmin=0, vmax=1)
Unfortunately, due to the way this errors I'm having a difficult time debugging it. I'm hoping someone with in-depth knowledge of masked_arrays will have some insight.
Exception exceptions.AttributeError: "'memmap' object has no attribute '_mmap'" in <bound method memmap.__del__ of memmap([ 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])> ignored
Some operations on numpy.memmap objects create new arrays, but unfortunately, the new array objects, which should be ndarrays, are created as numpy.memmap instances even though they aren't. When they go to clean up after themselves, they fail. A workaround would be to make numpy.memmap.__del__ more robust and do nothing if ._mmap isn't present. A real fix would be to figure out how to make sure that "memmap+memmap", etc., make ndarray instances rather than memmap instances. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (2)
-
Christopher Burns
-
Robert Kern