[Chicago] Kickstarter Fund to get rid of the GIL

David Beazley dxb251 at yahoo.com
Sat Jul 23 20:40:11 CEST 2011


> 
> "high performance just create multi processes that message" very rarely have
> I heard IPC and high performance in the same sentence.
> 
> Alex
> 

Your youth and inexperience is the only reason would make a statement that ignorant.   Go hang out with some people doing Python and supercomputing for awhile and report back---you will find that almost significant application is based on message passing (e.g., MPI).   This is because message passing has proven itself to be about the only sane way of scaling applications up to run across thousands to tens of thousands of CPU cores.

I speak from some experience as I was writing such software for large Crays, Connection Machines, and other systems when I first discovered Python back in 1996.   As early as 1995, our group had done performance experiments comparing threads vs. message passing on some multiprocessor SMP systems and found that threads just didn't scale or perform as well as message passing even on machines with as few as 4 CPUs.   This was all highly optimized C code for numerics (i.e., no Python or GIL).

That said, in order to code with message passing, you still need to know what you're doing and how to decompose your application to use it. 

Cheers,
Dave










More information about the Chicago mailing list