I'd really love to find a way to enable lazy loading by default, maybe with a way to opt-out old/problem/legacy modules instead of opt-in via __future__ or anything else. IME easily 95%+ of modules in the wild, today, will not even notice (I wrote an application bundler in past that enabled it globally by default without much fuss for several years). The only small annoyance is when it does cause problems the error can jump around, depending on how the lazy import was triggered.

module.__getattr__ works pretty well for normal access, after being imported by another module, but it doesn't properly trigger loading by functions defined in the module's own namespace. These functions are bound to module.__dict__ as their __globals__ so lazy loading of this variety is really dependant on a custom module.__dict__ that implements __getitem__ or __missing__.

I think this approach is the ideal path over existing PEPs. I've done it in the past and it worked very well. The impl looked something like this:

* Import statements and __import__ generate lazy-load marker objects instead of performing imports.
* Marker objects record the desired import and what identifiers were supposed to be added to namespace.
* module.__dict__.__setitem__ recognizes markers and records their identifiers as lazily imported somewhere, but **does not add them to namespace**.
* module.___getattribute__ will request the lazy attribute via module.__dict__ like regular objects and functions will request via their bound __globals__.
* Both will trigger module.__dict.__missing__, which looks to see if the requested identifier was previously marked as a lazy import, and if so, performs the import, saves to namespace properly, and returns the real import.


C Anthony

On Sep 10, 2017 1:49 PM, "Ivan Levkivskyi" <levkivskyi@gmail.com> wrote:
I have written a short PEP as a complement/alternative to PEP 549.
I will be grateful for comments and suggestions. The PEP should
appear online soon.



PEP: 562
Title: Module __getattr__
Author: Ivan Levkivskyi <levkivskyi@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017


It is proposed to support ``__getattr__`` function defined on modules to
provide basic customization of module attribute access.


It is sometimes convenient to customize or otherwise have control over
access to module attributes. A typical example is managing deprecation
warnings. Typical workarounds are assigning ``__class__`` of a module object
to a custom subclass of ``types.ModuleType`` or substituting ``sys.modules``
item with a custom wrapper instance. It would be convenient to simplify this
procedure by recognizing ``__getattr__`` defined directly in a module that
would act like a normal ``__getattr__`` method, except that it will be defined
on module *instances*. For example::

  # lib.py

  from warnings import warn

  deprecated_names = ["old_function", ...]

  def _deprecated_old_function(arg, other):

  def __getattr__(name):
      if name in deprecated_names:
          warn(f"{name} is deprecated", DeprecationWarning)
          return globals()[f"_deprecated_{name}"]
      raise AttributeError(f"module {__name__} has no attribute {name}")

  # main.py

  from lib import old_function  # Works, but emits the warning

There is a related proposal PEP 549 that proposes to support instance
properties for a similar functionality. The difference is this PEP proposes
a faster and simpler mechanism, but provides more basic customization.
An additional motivation for this proposal is that PEP 484 already defines
the use of module ``__getattr__`` for this purpose in Python stub files,
see [1]_.


The ``__getattr__`` function at the module level should accept one argument
which is a name of an attribute and return the computed value or raise
an ``AttributeError``::

  def __getattr__(name: str) -> Any: ...

This function will be called only if ``name`` is not found in the module
through the normal attribute lookup.

The reference implementation for this PEP can be found in [2]_.

Backwards compatibility and impact on performance

This PEP may break code that uses module level (global) name ``__getattr__``.
The performance implications of this PEP are minimal, since ``__getattr__``
is called only for missing attributes.


.. [1] PEP 484 section about ``__getattr__`` in stub files

.. [2] The reference implementation


This document has been placed in the public domain.

Python-ideas mailing list
Code of Conduct: http://python.org/psf/codeofconduct/