[Python-ideas] Python 3000 TIOBE -3%

[Terry Reedy]

On 2/9/2012 1:57 PM, Sturla Molden wrote:
> On 09.02.2012 19:42, Mike Meyer wrote:
>
>> If threading is the only acceptable concurrency mechanism, then Python
>> is the wrong language to use. But you're also not building scaleable
>> systems, which is most of where it really matters. If you're willing
>> to consider things other than threading - and you have to if you want
>> to build scaleable systems - then Python makes a good choice.
>
> Yes or no... Python is used for parallel computing on the biggest
> supercomputers, monsters like Cray and IBM blue genes with tens of
> thousands of CPUs. But what really fails to scale is the Python module
> loader! For example it can take hours to "import numpy" for 30,000
> Python processes on a blue gene.

Mike Meyer posted that on pydev today
http://mail.scipy.org/pipermail/numpy-discussion/2012-January/059801.html

They determined that the time was gobbled by *finding* modules in each 
process, so they cut hours by finding them in 1 process and sending the 
locations to the other 29,999. We are already discussing how to use this 
lesson in core Python. The sub-thread is today's posts in
"requirements for moving __import__ over to importlib?"

-- 
Terry Jan Reedy