On Sat, Feb 6, 2021 at 12:51 AM Matt del Valle <
matthewgdv@gmail.com> wrote:
>
> Unfortunately, [lazy importers are] fundamentally incapable of dealing with 'from x import y' syntax, because this is always loaded eagerly by the Python interpreter.
>
This is because it's fundamentally hard.
> With the increasingly widespread adoption of type hints I think it is time to revive the proposal for lazy imports as an official language feature.
>
Are you writing this proposal assuming PEP 563 behaviour? If type
hints are the primary justification, it may be less necessary. But
lazy importing does have other value.
> It also is fundamentally incapable of handling this other extremely common usage pattern:
>
> some_library/ __init__.py
>
> > __all__ = ["SomeClass", "AnotherClass"]
> >
> > from .extremely_expensive_module import SomeClass
> > from .another_extremely_expensive_module import AnotherClass
>
> some_module.py
>
> > from some_library import SomeClass
Hmm.
Okay. Let's talk semantics, not syntax. What exactly should the
interpreter do with this line? Is there an actual object in the module
dictionary under the name "SomeClass", and if so, what sort of object?
One possible meaning for this kind of "lazy from import" would be to
maintain a table of import definitions, just like the module
dictionary. (I would be inclined to say that this can ONLY be done at
module level; lazy imports inside a class or function seem less useful
and more hassle than they're worth. But if you disagree, feel free to
expand this to other namespaces.) Whenever a module name is looked up,
the interpreter looks first in the module dictionary, and then if it's
not found, tries the lazy imports; upon finding the import definition,
it removes it, performs the import, and stuffs the object into the
module dictionary.
This would be a fair amount of hassle, and it'd have a performance
impact on EVERY module name lookup (and everything that goes beyond
that to the builtins). But it would, I believe, give you an actual
lazy import system.
So the question is: how often is this going to be useful?
> Where a library author wants to make several classes in their library's public API available at package level in the __init__.py, but doesn't want to eagerly load all of them when many users will only ever use one in a given program. In this situation if I try to import SomeClass from some_library/__init__.py I will always trigger AnotherClass to be loaded as well, totally unnecessarily.
>
I wonder if this particular use-case is better served by a module
subclass that does the lazy loading. Effectively, you get to break up
a module into a package of N separately-loadable modules, and on first
use of anything from a particular shard, that shard gets loaded.
> This would be insanely useful. I really hope this sparks some discussion :)
Not sure HOW useful it would be in general, but yes, there definitely
are use-cases for it. I'm pretty dubious about having a separate
lookup on every name load just in case it's been lazily imported,
though.