Hi all, an odd problem that has bitten me a couple of times: I build an array from a memmap, the memmap goes out of scope and gets GC'ed (closed), then inexplicable data errors occur: numarray.libnumarray.error: NA_updateDataPtr: error getting read buffer data ptr The solution I came up with was to assign the memmap to a unique attribute of the array ('source_mmap_dont_gc_me'). Then it only gets GC'ed when all it's arrays are GC'ed. So, is it reasonable to make this standard in numarray: slices remember their memmaps and NumArrays remember their buffers (I assume the latter already happens). bye, Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
On Tue, 2005-05-10 at 11:12 +1000, Simon Burton wrote:
Hi all,
an odd problem that has bitten me a couple of times: I build an array from a memmap, the memmap goes out of scope and gets GC'ed (closed), then inexplicable data errors occur:
numarray.libnumarray.error: NA_updateDataPtr: error getting read buffer data ptr
The solution I came up with was to assign the memmap to a unique attribute of the array ('source_mmap_dont_gc_me'). Then it only gets GC'ed when all it's arrays are GC'ed.
So, is it reasonable to make this standard in numarray: slices remember their memmaps and NumArrays remember their buffers (I assume the latter already happens).
I think we should make MemmapSlices remember their parent Memmaps. The NumArray remembers the MemmapSlice (buffer) as long as it lives, and the slice in turn would keep the Memmap alive. I have hit a similar problem with Tkinter buttons, twice, so I know the aggravation involved. I'm nervous about sticking in a circular reference (the Memmap already remembers it's slices) but I suppose the garbage collector was added for exactly this case. I'm not entirely sure how hard it's going to be to add.
On Tue, 10 May 2005 08:13:34 -0400 Todd Miller <jmiller@stsci.edu> wrote:
I'm nervous about sticking in a circular reference (the Memmap already remembers it's slices) but I suppose the garbage collector was added for exactly this case. I'm not entirely sure how hard it's going to be to add.
Yes, this is a good point. But also, memmaps have a close method so users can always explicitly break this cycle if they want. Either way, appropriate warnings should be added to the documentation. bye, Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com
participants (2)
-
Simon Burton -
Todd Miller