[Numpy-discussion] Shared memory ndarrays (update)
sturla at molden.no
Mon Apr 11 15:29:35 EDT 2011
Den 11.04.2011 01:20, skrev Sturla Molden:
> - 64-bit support.
> - Memory leak on Linux/Unix should be gone (monkey patch for os._exit).
> - Added a global lock as there are callbacks to Python (the GIL is not
> sufficient serialization).
I will also add a barrier synchronization primitive to this (as it is
very useful for numerical computing, more so than mutexes or semaphores)
and a work scheduler, so it will be easy to distribute work between the
The barrier is implemented with atomic read/writes on top of shared
memory, so it can be sent over multiprocessing.Queue.
It is also possible to implement locks, semaphores, events, etc., the
Observe that the synchronization primitives in multiprocessing, e.g.
multiprocessing.RLock, cannot be sent over a Queue. Named shared memory
takes that restriction away. Thus, objects that are pickled for
multiprocessing.Queue can contain locks, events, barriers, etc.
More information about the NumPy-Discussion