[Python-Dev] Rethinking intern() and its data structure

Brendan Cully brendan at kublai.com
Sat Apr 11 02:52:01 CEST 2009


On Friday, 10 April 2009 at 15:05, P.J. Eby wrote:
> At 06:52 PM 4/10/2009 +1000, Nick Coghlan wrote:
>> This problem (slow application startup times due to too many imports at
>> startup, which can in turn can be due to top level imports for library
>> or framework functionality that a given application doesn't actually
>> use) is actually the main reason I sometimes wish for a nice, solid lazy
>> module import mechanism that manages to avoid the potential deadlock
>> problems created by using import statements inside functions.

I'd love to see that too. I imagine it would be beneficial for many
python applications.

> Have you tried http://pypi.python.org/pypi/Importing ? Or more  
> specifically, 
> http://peak.telecommunity.com/DevCenter/Importing#lazy-imports ?

Here's what we do in Mercurial, which is a little more user-friendly,
but possibly too magical for general use (but provides us a very nice
speedup):

http://www.selenic.com/repo/index.cgi/hg/file/tip/mercurial/demandimport.py#l1

It's nice and small, and it is invisible to the rest of the code, but
it's probably too aggressive for all users. The biggest problem is
probably that ImportErrors are deferred until first access, which
trips up modules that do things like

try:
  import foo
except ImportError
  import fallback as foo

of which there are a few. The mercurial module maintains a blacklist
as a bandaid, but it'd be great to have a real fix.


More information about the Python-Dev mailing list