[Numpy-discussion] .transpose() of memmap array fails to close()
peridot.faceted at gmail.com
Wed Aug 15 20:50:28 EDT 2007
On 15/08/07, Glen W. Mabey <Glen.Mabey at swri.org> wrote:
> On Tue, Aug 14, 2007 at 12:23:26AM -0400, Anne Archibald wrote:
> > On 13/08/07, Glen W. Mabey <Glen.Mabey at swri.org> wrote:
> > > As I have tried to think through what should be the appropriate
> > > behavior for the returned value of __getitem__, I have not been able to
> > > see an appropriate solution (let alone know how to implement it) to this
> > > issue.
> > Is the problem one of finalization? That is, making sure the memory
> > map gets (flushed and) closed exactly once? In this case the
> > numpythonic solution is to have only the original mmap object do any
> > finalization; any slices contain a reference to it anyway, so they
> > cannot be kept after it is collected. If the problem is that you want
> > to do an explicit close/flush on a slice object, you could just always
> > apply the close/flush to the base object of the slice if it has one or
> > the slice itself if it doesn't.
> The immediate problem is that when a numpy.memmap instance is created as
> another view of the original array, then __del__ on that new view fails.
Yes, this is definitely broken.
> flush()ing and closing aren't an issue for me, but they can't be
> performed at all on derived views right now. It seems to me that any
> derived view ought to be able to flush(), and ideally in my mind,
> close() would be called [automatically] only just before the reference
> count gets decremented to zero.
> That doesn't seem to match the numpythonic philosophy you described,
> Anne, but seems logical to me, while still allowing for both manual
> flush() and close() operations.
You have to be a bit careful, because a view really is just a view
into the array - the original is still around. So you can't really
delete the array contents when the view is deleted. Really, if you do:
B = A[::2]
nothing at all should happen to A.
But to be pythonic, or numpythonic, when the original A is
garbage-collected, the garbage collection should certainly close the
Being able to apply flush() or whatever to slices is not necessarily
unpythonic, but it's probably a lot simpler to reliably implement
slices of mmap()s as simple slices of ordinary arrays. It means you
need to keep the original mmap object around (or traverse up the tree
T = A
while T.base is not None: T = T.base
(Note that this would be simpler if when you did
A = arange(100)
B = A[::2]
C = B[::2]
you found that C.base were A rather than B.)
More information about the NumPy-Discussion