On Mon, 2003-11-24 at 14:13, Sebastian Haase wrote:
Sebastian Haase wrote:
Hi, Suppose I have a 500MB-ram Computer and a 300MB ram-only (standard) numarray. Now I would like to "save" that onto harddrive (with a small header
up
front
How about:
f = open(filename, 'wb') f.write(MyHeader) A.tofile(f)
To read it back in, you need to know where your header ends, by either parsing it or using one of the same size every time, then you can use fromfile() to create an array form it.
The main reason for my question was just to find out if NUMARRAY
supports
it, and how ? Also I have many "bookkeeping" functions already implemented for the memmap'd case. (That is, I have a class with member methods operting on a member
memmapped
array) So if what I descibed is possible I could save myself form duplicating
lot's
of code.
Essentially I was hoping for the most ellegant solution ;-)
memmap's Memmap class does support an insert() method for adding a new slice to the end of (or anywhere in) an existing map. The new slice, however, will exist as a block of memory allocated on the heap until the memmap is saved to disk.
Thus, two scenarios present themselves: (1) you allocate the new slice ahead of time and create an array from it, avoiding data duplication (2) you create an array and later copy it into (a newly inserted slice of) the memmap, thereby duplicating your data on the heap.
When you close the map, slices on the heap are written to the map file.
Todd
Thanks for your reply. Is it possible to define a memmap slice and giving it a (preinitialized !) memory buffer ? I'm thinking: I have a RAM-based numarray, I just take the buffer (pointer) and hand it over to the memmap-slice so that it can make the association between the disk-space and the RAM-space. I guess you are calling "heap" what I call RAM. Is memmap using something inherently different that heap? (I might be missing something...) As I was trying to illustrate in my example, my ram-numarray might already be using most of the available address space.
Thanks, Sebastian