[Python-Dev] Idea to support lazy loaded names.

[Brian Allen Vanderburg II]

I've got an idea for Python but don't know where to post it.  It seems
like this may be the right place.

When developing Python libraries (and any other language for that
matter), items are separated into modules for organization and other
purpose.  The following is an example fake framework to illustrate this:

  mini-framework/
    __init__.py
    app.py
      class BaseApplication
      class Application
      class ProxyApplication
    config.py
      class Config
      class ConfigLoader
    template.py
      class Template
      class TemplateCompiler
      class TemplateManager

In order to be used externally, one must reference the module:

  from miniframework.app import Application
  from miniframework.template import TemplateManager

This can be solved by importing these values directly in __init__.py.
However this requires fully loading those imported modules:

  __init__.py:
    from .app import BaseApplication, Application, ProxyApplication
    from .config import Config, ConfigLoader
    from .template import Template, TemplateCompiler, TemplateManager
  
One idea I had was to support lazy imported names.  I've seen some
frameworks implement this in various means, and figured the idea is good
to be part Python.

The idea is that for a new module attribute to exist: __lazyload__.
During the access of any attribute for a module, the following would
happen:

  * If the named attribute already exists, use it
  * If the named attribute does not already exist:
    * If a lazy load of the name has already been attempted, result in
      a NameError
    * If a lazy load of the name has not yet been attempted:
       * Check the __lazyload__ module attribute for the name, perform
         the loading operation and assign the module attribute the
         value if found or result in a NameError
       * Even if not found, set a flag that lazy load has been
         attempted so it will not be attempted again for the same name

The __lazyload__ attribute is intended to be a dictionary.  The key of
the dictionary is the name of the attribute that would be set/tested
for in the module.  The value of the dictionary is a string that
represents either the module name to load or the module name and
attribute to load.  If the value starts with a dot, then it is treated
as a relative import relative to the module/package containing the
__lazyload__ value.

With this idea, the packages __init__.py file could look like this:

  __lazyload__ = {
    'BaseApplication': '.app.BaseApplication',
    'Application': '.app.Application',
    ...
  }
       
The end use of the package (and even the developer) can then perform an
import as follows:

  from miniframework import Application

instead of:

  from miniframework.app import Application

This allows the public api be be cleaner, while still being efficient
by not loading all modules in __init__.py until the value is actually
accessed.



Brian Allen Vanderburg II