On 25/06/15 02:09, Devin Jeanpierre wrote:
Although, threads are better for memory usage, by defaulting to sharing even on write. (Good for memory, maybe not so good for bug-freedom...)
I am not sure. Code written to use OpenMP tend to have less bugs than code written to use MPI. This suggests that shared memory is easier than message-passing, which is contrary to the common belief. My own experience with OpenMP and MPI suggests it is easier to create a deadlock with message-passing than accidentally have threads access the same address concurrently. This is also what I hear from other people who writes code for scientific computing. I see a lot of claims that message-passing is supposed to be "safer" than a shared memory model, but that is not what we see with OpenMP and MPI. With MPI, the programmer must make sure that the send and receive commands are passed in the right order at the right time, in each process. This leaves plenty of room for messing up or creating unmaintainable spaghetti code, particularly in a complex algorithm. It is easier to make sure all shared objects are protected with mutexes than to make sure a spaghetti of send and receive messages are in correct order. It might be that Python's queue method of passing messages leave less room for deadlocking than the socket-like MPI_send and MPI_recv functions. But I think message-passing are sometimes overrated as "the safe solution" to multi-core programming (cf. Go and Erlang). Sturla