[Numpy-discussion] Annoyance of memap rraywithmultiprocessing.Pool.applay_async

[Nadav Horesh]

Is there a way to use memory mapped files as if they were shared memory? I made an application in which some (very often non contiguous) parts of a memmap array are processed by different processors. However I might use shared memory array instead. I wonder, since both types share common properties, if there a way to interchange then transparently.


  Nadav

-----Original Message-----
From: numpy-discussion-bounces at scipy.org on behalf of Robert Kern
Sent: Sun 04-Apr-10 18:45
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] Annoyance of memap rraywithmultiprocessing.Pool.applay_async
 
On Sat, Apr 3, 2010 at 22:35, Nadav Horesh <nadavh at visionsense.com> wrote:
> Got it, thank you.
> But why, nevertheless, the results are correct although the pickling is impossible?

Rather, I meant that they don't pickle correctly. They use ndarray's
pickling, which will copy the data, and then reconstruct an ndarray on
the other side and just change the type to memmap without actually
memory-mapping the file. Thus you have a __del__ method referring to
attributes that haven't been set up.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion

-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 3824 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100405/e21fa263/attachment.bin>