[Numpy-discussion] passing arrays between processes

[Dag Sverre Seljebotn]

Bryan Cole wrote:
> I'm starting work on an application involving cpu-intensive data
> processing using a quad-core PC. I've not worked with multi-core systems
> previously and I'm wondering what is the best way to utilise the
> hardware when working with numpy arrays. I think I'm going to use the
> multiprocessing package, but what's the best way to pass arrays between
> processes?

You may want to look at MPI, e.g. mpi4py is convenient for this kind of 
work. For numerical work across processes it is close to a de facto 
standard.

It requires an MPI implementation set up on your machine though (but for 
single-machine use this isn't hard to set up, typically just install 
e.g. OpenMPI), and that you launch Python through

mpiexec -n 4 python somescript.py

-- 
Dag Sverre