[Numpy-discussion] mmap (was: fast constructor for arrays from byte data)
Phil Austin
phil at geog.ubc.ca
Wed Aug 8 16:56:56 EDT 2001
Chris Barker writes:
> I don't understand mmap at all. From the name, it sounds like the entire
> contents of the file is mapped into memory, so the memory would get used
> as soon as you set it up. If anyone knows, I'd like to hear...
This might be more than you need to know, but there's a
copy of the article:
"Why aren't you using mmap() yet?"
by Kevin Sheehan at:
http://thrush.eos.ubc.ca/users/phil/mmap/mmap.pdf (note reversed page
order)
and a Solaris/SGI email debate in mhonarc format:
http://thrush.eos.ubc.ca/users/phil/mmap/threads.html
bottom line: you really need a vm implementation that supports
madvise() to get the maximum benefit from mmap. (Linux 2.4
might have madvise, I haven't tried it out yet -- 2.2 didn't).
The way I used to use mmap was through a numpy array class that was
initiated from a pointer to a mmap'd file. This made reading/writing
to the file as simple as indexing into the numpy array. I haven't
been back to reimplement that class on linux since I left solaris,
though. You can also get some of the benefits of mmap via netcdf,
which will use mmap if it exists.
Regards, Phil
More information about the NumPy-Discussion
mailing list