Parallelization in Python 2.6
sturlamolden at yahoo.no
Wed Aug 19 14:39:15 CEST 2009
On 19 Aug, 05:27, Dave Angel <da... at ieee.org> wrote:
> But if you do it that way, it's slower than sequential. And if you have
> a multi-core processor, or two processors, or ... then it gets much
> slower yet, and slows down other tasks as well.
> With the current GIL implementation, for two CPU-bound tasks, you either
> do them sequentially, or make a separate process.
For CPU bound tasks, one should put the bottleneck in C/Fortran/Cython
and release the GIL. There is a speed penalty of 200x from using
Python instead of C. With a quadcore processor you can gain less than
4x speed-up by parallelizing. If you really care enough about speed to
write parallel code, the first thing you should do is migrate the
bottleneck to C.
More information about the Python-list