[Numpy-discussion] multiprocessing shared arrays and numpy
Nadav Horesh
nadavh at visionsense.com
Thu Mar 4 08:06:34 EST 2010
Extended module that I used for some useful work.
Comments:
1. Sturla's module is better designed, but did not work with very large (although sub GB) arrays
2. Tested on 64 bit linux (amd64) + python-2.6.4 + numpy-1.4.0
Nadav.
-----Original Message-----
From: numpy-discussion-bounces at scipy.org on behalf of Nadav Horesh
Sent: Thu 04-Mar-10 11:55
To: Discussion of Numerical Python
Subject: RE: [Numpy-discussion] multiprocessing shared arrays and numpy
Maybe the attached file can help. Adpted and tested on amd64 linux
Nadav
-----Original Message-----
From: numpy-discussion-bounces at scipy.org on behalf of Nadav Horesh
Sent: Thu 04-Mar-10 10:54
To: Discussion of Numerical Python
Subject: Re: [Numpy-discussion] multiprocessing shared arrays and numpy
There is a work by Sturla Molden: look for multiprocessing-tutorial.pdf
and sharedmem-feb13-2009.zip. The tutorial includes what is dropped in
the cookbook page. I am into the same issue and going to test it today.
Nadav
On Wed, 2010-03-03 at 15:31 +0100, Jesper Larsen wrote:
> Hi people,
>
> I was wondering about the status of using the standard library
> multiprocessing module with numpy. I found a cookbook example last
> updated one year ago which states that:
>
> "This page was obsolete as multiprocessing's internals have changed.
> More information will come shortly; a link to this page will then be
> added back to the Cookbook."
>
> http://www.scipy.org/Cookbook/multiprocessing
>
> I also found the code that used to be on this page in the cookbook but
> it does not work any more. So my question is:
>
> Is it possible to use numpy arrays as shared arrays in an application
> using multiprocessing and how do you do it?
>
> Best regards,
> Jesper
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 6120 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100304/9353fd57/attachment.bin>
More information about the NumPy-Discussion
mailing list