Since this hasn't actually received moderator approval yet, I'd be happy if you declined this post and let me resubmit it without those typos, since they actually deal with specifics of the proposal and could be confusing if left in. I'm super sorry, my mistake!
On Fri, Feb 5, 2021 at 12:39 PM Matt del Valle firstname.lastname@example.org wrote:
Whoops, sorry. I just realized that I mistyped the third paragraph from the bottom, it should be:
from some_library import AnotherClass # this causes * AnotherClass * to
be imported from some_library/__init__.py which in turn causes the import statement on line 4 of some_library/__init__.py to finally happen for real. The import of *SomeClass* on line 3 *still* hasn't actually happened, and may never happen if *SomeClass* isn't imported (or lazy imported and then used) during the lifetime of this program.
On Fri, Feb 5, 2021 at 12:34 PM Matt del Valle email@example.com wrote:
I have seen a lot of interest in this topic, and I think the sheer number of different lazy import implementations (generally using proxy objects for modules and then loading them when an attribute is accessed) should present pretty good evidence that this is a very desired feature, particularly for writers of library code seeking to optimize start-up times and similar.
Unfortunately, all of these implementations have serious limitations in that often static code analysis tools cannot understand them, resulting in autocompletions and useful diagnostic errors being lost. Also, they are fundamentally incapable of dealing with 'from x import y' syntax, because this is *always *loaded eagerly by the Python interpreter.
With the increasingly widespread adoption of type hints I think it is time to revive the proposal for lazy imports as an official language feature.
It is very common to type hint the return type or argument type of a function as a type which cannot be imported at the top of the file inside the module namespace because it would trigger a circular import, and must therefore be imported inside the function. In these instances it is currently recommended to use the 'TYPE_CHECKING' value in the 'typing' module which is always 'False' at runtime, in order to be able to provide the return or argument type hint:
from typing import TYPE_CHECKING
if TYPE_CHECKING: from module_that_depends_on_me import MyClass
def some_func() -> MyClass: from module_that_depends_on_me import MyClass
However, this is unnecessarily verbose and confusing to most users who are not familiar with this usage pattern. IDEs will also be unable to warn you that MyClass is undefined if you try to use it inside 'some_func' without importing it within 'some_func' first, because it appears to be defined even though it isn't.
It also is fundamentally incapable of handling this other extremely common usage pattern:
__all__ = ["SomeClass", "AnotherClass"]
from .extremely_expensive_module import SomeClass from .another_extremely_expensive_module import AnotherClass
from some_library import SomeClass
Where a library author wants to make several classes in their library's public API available at package level in the __init__.py, but doesn't want to eagerly load all of them when many users will only ever use one in a given program. In this situation if I try to import SomeClass from some_library/__init__.py I will *always* trigger AnotherClass to be loaded as well, totally unnecessarily.
My proposed syntax is pretty much the same as was suggested by Nicolas Cellier in 2017 here: https://mail.python.org/pipermail/python-ideas/2017-February/044894.html, but with the additional stipulation that 'from x import y' format imports should *also* be supported. The two new lazy import statements should therefore be:
lazy import some_module from some_module lazy import SomeClass
For the two examples given above the equivalent code should be.
from module_that_depends_on_me lazy import MyClass # does not raise
ImportError when module_that_depends_on_me tries to import me, because this import is deferred
def some_func() -> MyClass: return MyClass() # MyClass is not imported until this line, in a
function that isn't called until after module level imports have already been resolved
__all__ = ["SomeClass", "AnotherClass"]
from .extremely_expensive_module lazy import SomeClass # this import is
from .another_extremely_expensive_module lazy import AnotherClass #
this import is deferred
from some_library import AnotherClass # this causes SomeClass to be
imported from some_library/__init__.py which in turn causes the import statement on line 4 of some_library/__init__.py to finally happen for real. The import of AnotherClass on line 3 *still* hasn't actually happened, and may never happen if AnotherClass isn't imported (or lazy imported and then used) during the lifetime of this program.
These are just some examples that I could think of off the top of my head. I am certain there are more.
This would be insanely useful. I really hope this sparks some discussion :)