using mmap on large (> 2 Gig) files

sturlamolden sturlamolden at
Tue Oct 24 02:57:50 CEST 2006

myeates at wrote:
> Hi
> Anyone ever done this? It looks like Python2.4 won't take a length arg
> > 2 Gig since its not seen as an int.

Looking at Python's source (mmapmodule.c), it seems that "mmap.mmap"
always sets the offset argument in Windows' MapViewOfFile and UNIX'
mmap to 0. This means that it is always mapping from the beginning of
the file. Thus, Python's mmap module is useless for large files. This
is really bad coding. The one that wrote mmapmodule.c didn't consider
the possibility that a 64 bit file system like NTFS can harbour files
to large to fit in a 32 address space. Thus, mmapmodule.c needs to be
fixed before it can be used for large files.

More information about the Python-list mailing list