using mmap on large (> 2 Gig) files
sturlamolden at yahoo.no
Tue Oct 24 02:57:50 CEST 2006
myeates at jpl.nasa.gov wrote:
> Anyone ever done this? It looks like Python2.4 won't take a length arg
> > 2 Gig since its not seen as an int.
Looking at Python's source (mmapmodule.c), it seems that "mmap.mmap"
always sets the offset argument in Windows' MapViewOfFile and UNIX'
mmap to 0. This means that it is always mapping from the beginning of
the file. Thus, Python's mmap module is useless for large files. This
is really bad coding. The one that wrote mmapmodule.c didn't consider
the possibility that a 64 bit file system like NTFS can harbour files
to large to fit in a 32 address space. Thus, mmapmodule.c needs to be
fixed before it can be used for large files.
More information about the Python-list