[Numpy-discussion] np.memmap and memory usage

[Emmanuelle Gouillart]

	Hello,

	I'm using numpy.memmap to open big 3-D arrays of Xray tomography
data. After I have created a new array using memmap, I modify the
contrast of every Z-slice (along the first dimension) inside a for loop,
for a better visualization of the data. Although I call memmap.flush
after each modification of a Z-slice, the memory used by Ipython keeps
increasing at every new iteration. At the end of the loop, the memory
used by Ipython is of the order of magnitude of the size of the data file
(1.8Go !). I would have expected that the maximum amount of memory used
would corresponde to only one Z-slice of the 3D array. See the code
snapshots below for more details.

	Is this an expected behaviour? How can I reduce the amount of
memory used by Ipython and still process my data?

import numpy as np
imin, imax = -2, 2
data = np.memmap(filename = 'data.vol', mode='r+', dtype=float32,
shape=(512,981, 981))
for sl in data:
    sl[sl<imin] = imin
    sl[sl>imax] = imax
    data.flush() # tried with and without this line

	Thanks in advance,

	Emmanuelle