[Python-ideas] lazy import via __future__ or compiler analysis

Neil Schemenauer nas-python-ideas at arctrix.com
Fri Sep 8 12:19:01 EDT 2017

On 2017-09-08, Joshua Morton wrote:
> In general this won't work. It's not generally possible to know if a given
> statement has side effects or not.

That's true but with the AST static analysis, we find anything that
has potential side effects.  The question if any useful subset of
real modules pass these checks.  If we flag everything as no lazy
import safe then we don't gain anything.

> As an example, one normally wouldn't expect function or class
> definition to have side effects, but if a function is decorated,
> the decorators are evaluated at function "compilation"/import
> time, and may have side effects.

Decorators are handled in my latest prototype (module is not lazy).

>  As another example, one can put arbitrary expressions in a
>  function annotation, and those are evaluated at import time.

Not handled yet but no reason they can't be.

> As a result of this, you can't even know if an import is safe,
> because that module may have side effects. That is, the module
> foo.py:
>     import bar
> isn't known to be lazy, because bar may import and start the logging
> module, as an example.

That is handled as well.  We only need to know if the current module
is lazy safe or not.  Imports of submodules that have side-effects
will have those side effects happen like they do now.

The major challenge I see right now is 'from .. import' and class
bases (i.e. metaclass behavior).  If we do the safe thing then all
from-imports make the module unsafe for lazy loading and any class
definition that has a base class is also unsafe.

I think the idea is not yet totally dead though.  We could have a
command-line option to enable it.  Modules that depend on
side-effects of from-import and from base classes could let the
compiler know about that somehow (make it explicit).  That would
also a good fraction of modules to be lazy import safe.



More information about the Python-ideas mailing list