[New-bugs-announce] [issue17038] multiprocessing only use one core when C module are imported
report at bugs.python.org
Sat Jan 26 10:39:23 CET 2013
New submission from HadiM:
This is the first time I report a bug so if I did something wrong please let me know. I also tried to ask on #python and #python-fr before posting here to know if it was a bug or a possible multiprocessing limitation on Linux. So I'm sorry if it's not a bug...
The problem appears when a module with .c compiled file is imported (sush as numpy, scipy, pandas, tables, skimage, I did not tested with a custom .c module). Multiprocessing and more specifically imap and map method of Pool class did not distribute processes along all the core but only on one core. This problem does not appears when I don't import a module with .c compiled.
I know distributing processes along cores is specific to the OS implementation and not related to Python but I said that to illustrate the behaviour.
I should notice that I did not see the bug with a Windows 7, 32 bits (Virtualbox).
My laptop run with the latest ubuntu 64bits (tested with both kernel 3.5 and 3.8 last rc). It's a classic Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz with 4 cores.
You can try to reproduce the bug with the attached file. You can also find the same file here : http://pastebin.com/jqq9G5Ph
Thank you for your time !
components: Library (Lib), ctypes
title: multiprocessing only use one core when C module are imported
versions: Python 2.7
Added file: http://bugs.python.org/file28841/bug.py
Python tracker <report at bugs.python.org>
More information about the New-bugs-announce