[Numpy-discussion] Views of memmaps and offset

Charles R Harris charlesr.harris at gmail.com
Sat Sep 22 12:15:52 EDT 2012


On Sat, Sep 22, 2012 at 9:46 AM, Olivier Grisel <olivier.grisel at ensta.org>wrote:

> There is also a third use case that is problematic on numpy master:
>
> orig = np.memmap('tmp.mmap', dtype=np.float64, shape=100, mode='w+')
> orig[:] = np.arange(orig.shape[0]) * -1.0  # negative markers to
> detect under / overflows
>
> a = np.memmap('tmp.mmap', dtype=np.float64, shape=50, mode='r+', offset=16)
> a[:] = np.arange(50)
> b = np.asarray(a[10:])
>
> Now b does not even have a 'filename' attribute anymore. `b.base` is a
> python mmap instance but the later is created with a file descriptor.
>
> It would still be possible to use:
>
> from _multiprocessing import address_of_buffer
>
> to find the memory address of the mmap buffer and use than to open new
> buffer views on the same memory segment from subprocesses using
> `numpy.frombuffer((ctypes.c_byte * n_byte).fromaddress(addr))` but in
> case of failure (e.g. the file has been deleted on the HDD) one gets a
> segmentation fault instead of a much more userfriendly catchable file
> not found exception.
>

Would some sort of 'dup' method be useful?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20120922/eb31bc53/attachment.html>


More information about the NumPy-Discussion mailing list