before posting to python-dev I thought is is the best to discuss this here. And I assume that someone else had the same idea then me before. Maybe you can point me to the relevant discussion/ticket.
I read about Intels hybrid CPUs. It means there are multiple cores e.g. 8 high-speed cores and 8 low-speed (but more energy efficient) cores combined in one CPU.
In my use cases I do parallelize with Pythons multiprocessing package to work on millions of rows on pandas.DataFrame objects. This are task that are not vecotrizable. I simple cut the DataFrame horizontal in pieces (numbered by the available cores).
But when the cores are different in there "speed" I need to know that. e.g. with a 16 core CPU where half of the cores low/slow and every core has 1 million rows to work on. The 8 high speed cores are finishing earlier and just waiting untill the slow cores are finished. It would be more efficient if the 8 high speed cores each would work on 1,3 million rows and the low speed cores each on 0,7 million rows. It is not perfect but better. I know that they will not finish all at the same timepoint. But their end time will be closer together.
But to do this I need to know the type of the cores.
Am I wrong?
Are there any plans in the Python development taking this into account?