[Python-Dev] To reduce Python "application" startup time
brett at python.org
Wed Sep 20 18:50:28 EDT 2017
I should mention that I have a prototype design for improving importlib's
lazy loading to be easier to turn on and use. See
for my current notes. Part of it includes an explicit lazy_import()
function which would negate needing to hide imports in functions to delay
On Wed, 6 Sep 2017 at 20:50 INADA Naoki <songofacandy at gmail.com> wrote:
> > I’m not sure however whether burying imports inside functions (as a kind
> of poor man’s lazy import) is ultimately going to be satisfying. First,
> it’s not natural, it generally violates coding standards (e.g. PEP 8), and
> can make linters complain.
> Of course, I tried to move imports only when (1) it's only used one
> or two of many functions in the module,
> (2) it's relatively heavy, (3) it's rerely imported from other modules.
> > Second, I think you’ll end up chasing imports all over the stdlib and
> third party modules in any sufficiently complicated application.
> Agree. I won't use much time to optimization by moving import from
> top to inner function in stdlib.
> I think my import-profiler patch can be polished and committed in Python
> to help
> library maintainers to know import time easily. (Maybe, `python -X
> > Third, I’m not sure that the gains you’ll get won’t just be overwhelmed
> by lots of other things going on, such as pkg_resources entry point
> processing, pth file processing, site.py effects, command line processing
> libraries such as click, and implicitly added distribution exception hooks
> (e.g. Ubuntu’s apport).
> Yes. I noticed some of them while profiling imports.
> For example, old-style namespace package imports types module for
> Types module imports functools, and functools imports collections.
> So some efforts in CPython (Avoid importing collections and functools
> from site) is not
> worth enough when there are at least one old-style namespace package
> is installed.
> > Many of these can’t be blamed on Python itself, but all can contribute
> significantly to Python’s apparent start up time. It’s definitely worth
> investigating the details of Python import, and a few of us at the core
> sprint have looked at those numbers and thrown around ideas for
> improvement, but we’ll need to look at the effects up and down the stack to
> improve the start up performance for the average Python application.
> Yes. I totally agree with you. That's why I use import-profile.patch
> for some 3rd party libraries.
> Currently, I have these ideas to optimize application startup time.
> * Faster, or lazily compiling regular expression. (pkg_resources
> imports pyparsing, which has lot regex)
> * More usable lazy import. (which can be solved "PEP 549: Instance
> Properties (aka: module properties)")
> * Optimize enum creation.
> * Faster namedtuple (There is pull request already)
> * Faster ABC
> * Breaking large import tree in stdlib. (PEP 549 may help this too)
> INADA Naoki <songofacandy at gmail.com>
> Python-Dev mailing list
> Python-Dev at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Python-Dev