[Numpy-discussion] Adding `offset` argument to np.lib.format.open_memmap and np.load

Robert Kern robert.kern at gmail.com
Mon Feb 28 20:10:17 EST 2011


On Mon, Feb 28, 2011 at 18:50, Sturla Molden <sturla at molden.no> wrote:
> Den 01.03.2011 01:15, skrev Robert Kern:
>> You can have each of those processes memory-map the whole file and
>> just operate on their own slices. Your operating system's virtual
>> memory manager should handle all of the details for you.
>
> Mapping large files from the start will not always work on 32-bit
> systems. That is why mmap.mmap take an offset argument now (Python 2.7
> and 3.1.)
>
> Making a view np.memmap with slices is useful on 64-bit but not 32-bit
> systems.

I'm talking about the OP's stated use case where he generates the file
via memory-mapping the whole thing on the same machine. The whole file
does fit into the address space in his use case.

I'd like to see a real use case where this does not hold. I suspect
that this is not the API we would want for such use cases.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco



More information about the NumPy-Discussion mailing list