[Python-ideas] lazy import via __future__ or compiler analysis

[Joshua Morton]

Replying here, although this was written in response to the other thread:

Hey Neil,

In general this won't work. It's not generally possible to know if a given
statement has side effects or not. As an example, one normally wouldn't
expect function or class definition to have side effects, but if a function
is decorated, the decorators are evaluated at function "compilation"/import
time, and may have side effects. As another example, one can put arbitrary
expressions in a function annotation, and those are evaluated at import
time.

As a result of this, you can't even know if an import is safe, because that
module may have side effects. That is, the module foo.py:

    import bar

isn't known to be lazy, because bar may import and start the logging
module, as an example.

While in general you might think that global state modifying decorators or
annotations are a bad idea, they are used (Flask). As a result, it's not
possible to implement this without breaking behavior for certain users.
While I'm not a core dev and thus can't say with certainty, I will say that
that makes it very unlikely for this change (or others like it) to be
implemented.

It might be (from a language standpoint) to implement a `lazy` keyword (ie.
`lazy import foo; lazy def bar(): ...`), but if I recall, there's been
discussion of that and it's never gotten very far.

--Josh

On Thu, Sep 7, 2017 at 1:46 PM Neil Schemenauer <nas at arctrix.com> wrote:

> Barry Warsaw <barry at python.org> wrote:
> > There are a few other things that might end up marking a module as
> > "industrious" (my thesaurus's antonym for "lazy").
>
> Good points.  The analysis can be simple at first and then we can
> enhance it to be smarter about what is okay and still lazy load.  We
> may evolve it over time too, making things that are not strictly
> safe still not trigger the "industrious" load lazy anyhow.
>
> Another idea is to introduce __lazy__ or some such in the global
> namespace of the module, if present, e.g.
>
> __lazy__ = True
>
> then the analysis doesn't do anything except return True.  The
> module has explicitly stated that side-effects in the top-level code
> are okay to be done in a lazy fashion.
>
> Perhaps with a little bit of smarts in the analsis and a little
> sprinkling of __lazy__ flags, we can get a big chunk of modules to
> lazy load.
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20170908/9ba91bf3/attachment.html>