[Python-ideas] lazy import via __future__ or compiler analysis

[Neil Schemenauer]

This is a half baked idea that perhaps could work.  Maybe call it
2-stage module load instead of lazy.

Introduce a lazy module import process that modules can opt-in to.
The opt-in would either be with a __future__ statement or the
compiler would statically analyze the module and determine if it is
safe.  E.g. if the module has no module level statements besides
imports.

.pyc files get some other bits of information:

A) whether the module has opted for lazy import (IS_LAZY)

B) the modules imported by the module (i.e. top-level imports,
IMPORT_LIST)

Make __import__ understand this data and do lazy loading for modules
that want it.  Sub-modules that have import side-effects will still
execute as normal and the side effects will happen when the parent
module is imported..

This would consist of a recursive process, something like:

def load_module(name):
    if not IS_LAZY(name):
        import as usual
    else:
        create lazy version of module 'name'
        for subname in IMPORT_LIST(name):
            load_module(subname)
    
An additional idea from Barry W, if a module wants lazy loading but
wants to do some init when the module is "woken up", define a
__init__ top-level function.  Python would call that function when
attributes of the module are first actually used.

My plan was to implement this with a Python __import__
implementation. I would unmarshal the .pyc, compute IS_LAZY and
IMPORT_LIST at import time.  So, not gaining a lot of speedup.  It
would prove if the idea works in terms of not causing application
crashes, etc.  I could try running it with bigger apps and see how
many modules are flagged for lazy loading.