using mmap on large (> 2 Gig) files
donn at u.washington.edu
Tue Oct 24 18:22:51 CEST 2006
In article <1161648415.830523.225520 at k70g2000cwa.googlegroups.com>,
"sturlamolden" <sturlamolden at yahoo.no> wrote:
> It seems that Python does take a length argument, but not an offset
> argument (unlike the Windows' CreateFileMapping/MapViewOfFile and UNIX'
> mmap), so you always map from the beginning of the file. Of course if
> you have ever worked with memory mapping files in C, you will probably
> have experienced that mapping a large file from beginning to end is a
> major slowdown.
I certainly have not experienced that. mmap itself takes nearly
no time, there should be no I/O. Access to mapped pages may
require I/O, but there is no way around that in any case.
> I haven't looked at the source, but I'd be surprised if Python actually
> maps the file into the process image when mmap is called. I believe
> Python is not memory mapping at all; rather, it just opens a file in
> the file system and uses fseek to move around.
Wow, you're sure a wizard! Most people would need to look before
making statements like that.
Donn Cave, donn at u.washington.edu
More information about the Python-list