[Python-ideas] PEP 562

[C Anthony Risinger]

I'd really love to find a way to enable lazy loading by default, maybe with
a way to opt-out old/problem/legacy modules instead of opt-in via
__future__ or anything else. IME easily 95%+ of modules in the wild, today,
will not even notice (I wrote an application bundler in past that enabled
it globally by default without much fuss for several years). The only small
annoyance is when it does cause problems the error can jump around,
depending on how the lazy import was triggered.

module.__getattr__ works pretty well for normal access, after being
imported by another module, but it doesn't properly trigger loading by
functions defined in the module's own namespace. These functions are bound
to module.__dict__ as their __globals__ so lazy loading of this variety is
really dependant on a custom module.__dict__ that implements __getitem__ or
__missing__.

I think this approach is the ideal path over existing PEPs. I've done it in
the past and it worked very well. The impl looked something like this:

* Import statements and __import__ generate lazy-load marker objects
instead of performing imports.
* Marker objects record the desired import and what identifiers were
supposed to be added to namespace.
* module.__dict__.__setitem__ recognizes markers and records their
identifiers as lazily imported somewhere, but **does not add them to
namespace**.
* module.___getattribute__ will request the lazy attribute via
module.__dict__ like regular objects and functions will request via their
bound __globals__.
* Both will trigger module.__dict.__missing__, which looks to see if the
requested identifier was previously marked as a lazy import, and if so,
performs the import, saves to namespace properly, and returns the real
import.

-- 


C Anthony


On Sep 10, 2017 1:49 PM, "Ivan Levkivskyi" <levkivskyi at gmail.com> wrote:

I have written a short PEP as a complement/alternative to PEP 549.
I will be grateful for comments and suggestions. The PEP should
appear online soon.

--
Ivan

***********************************************************

PEP: 562
Title: Module __getattr__
Author: Ivan Levkivskyi <levkivskyi at gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 09-Sep-2017


Abstract
========

It is proposed to support ``__getattr__`` function defined on modules to
provide basic customization of module attribute access.


Rationale
=========

It is sometimes convenient to customize or otherwise have control over
access to module attributes. A typical example is managing deprecation
warnings. Typical workarounds are assigning ``__class__`` of a module object
to a custom subclass of ``types.ModuleType`` or substituting ``sys.modules``
item with a custom wrapper instance. It would be convenient to simplify this
procedure by recognizing ``__getattr__`` defined directly in a module that
would act like a normal ``__getattr__`` method, except that it will be
defined
on module *instances*. For example::

  # lib.py

  from warnings import warn

  deprecated_names = ["old_function", ...]

  def _deprecated_old_function(arg, other):
      ...

  def __getattr__(name):
      if name in deprecated_names:
          warn(f"{name} is deprecated", DeprecationWarning)
          return globals()[f"_deprecated_{name}"]
      raise AttributeError(f"module {__name__} has no attribute {name}")

  # main.py

  from lib import old_function  # Works, but emits the warning

There is a related proposal PEP 549 that proposes to support instance
properties for a similar functionality. The difference is this PEP proposes
a faster and simpler mechanism, but provides more basic customization.
An additional motivation for this proposal is that PEP 484 already defines
the use of module ``__getattr__`` for this purpose in Python stub files,
see [1]_.


Specification
=============

The ``__getattr__`` function at the module level should accept one argument
which is a name of an attribute and return the computed value or raise
an ``AttributeError``::

  def __getattr__(name: str) -> Any: ...

This function will be called only if ``name`` is not found in the module
through the normal attribute lookup.

The reference implementation for this PEP can be found in [2]_.


Backwards compatibility and impact on performance
=================================================

This PEP may break code that uses module level (global) name
``__getattr__``.
The performance implications of this PEP are minimal, since ``__getattr__``
is called only for missing attributes.


References
==========

.. [1] PEP 484 section about ``__getattr__`` in stub files
   (https://www.python.org/dev/peps/pep-0484/#stub-files)

.. [2] The reference implementation
   (https://github.com/ilevkivskyi/cpython/pull/3/files)


Copyright
=========

This document has been placed in the public domain.

_______________________________________________
Python-ideas mailing list
Python-ideas at python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170910/7d52f2c8/attachment-0001.html>