Wim Bakker wrote:
For a long time now, numpy's memmap has me puzzled by its behavior. When I use memmap straightforward on a file it seems to work fine, but whenever I try to do a memmap using a dtype it seems to gobble up the whole file into memory.
I don't understand your question. From my experience, the memmap is working fine. Please post and example that illustrates your point.
This, of course, makes the use of memmap futile. I would expect that the result of such an operation would give me a true memmap and that the data would be converted to dtype on the fly.
There is no conversion on the fly when you use memmap. You construct an array of the same data-type as is in the file and then manipulate portions of it as needed.
Am I doing something wrong? Are my expectations wrong?
My guess is that your expectations are not accurate, but example code would help sort it out.