[Python-ideas] Python 3000 TIOBE -3%
sturla at molden.no
Thu Feb 9 20:23:14 CET 2012
On 09.02.2012 20:05, Guido van Rossum wrote:
> I'm curious about the module loader problem. Did someone ever analyze
> the cause and come up with a fix? Is it the import lock? Maybe it's
> something for the bug tracker.
The offender is actually imp.find_module, which results in huge number
of failed open() calls when used concurrently from many processes.
So a solution is to have one process locate the modules and then
broadcast their location to the other processes.
There is even a paper on the issue. Here they suggest importing from
ramdisk might work on IBM blue gene, but not on Cray.
Another solution might be to use sys.meta_path to bypass imp.find_module:
The best solution would of course be to fix imp.find_module so it scales
More information about the Python-ideas