[Python-Dev] To reduce Python "application" startup time

[Chris Angelico]

On Wed, Sep 6, 2017 at 2:30 PM, INADA Naoki <songofacandy at gmail.com> wrote:
>>> This patch moves a few imports inside functions. I wonder whether that kind
>>> of change actually helps with real applications—doesn't any real application
>>> end up importing the socket module anyway at some point?
>>
>> I don't know if this particular change is worthwhile, but one place
>> where startup slowness is particularly noticed is with commands like
>> 'foo.py --help' or 'foo.py --generate-completions' (the latter called
>> implicitly by hitting <tab> in some shell), which typically do lots of
>> imports that end up not being used.
>>
>
> Yes. And There are more worse scenario.
>
> 1. Jinja2 supports asyncio.  So it imports asyncio.
> 2. asyncio imports concurrent.futures, for compatibility with Future class.
> 3. concurrent.futures package does
>     `from concurrent.futures.process import ProcessPoolExecutor`
> 4. concurrent.futures.process package imports multiprocessing.
>
> So when I use Jinja2 but not asyncio or multiprocessing, I need to import
> large dependency tree.
> I want to make `import asyncio` dependency tree smaller.
>
> FYI, current version of Jinja2 has very large regex which took more than 100ms
> when import time.  It is fixed in master branch.  So if you try to see
> Jinja2, please
> use master branch.

How significant is application startup time to something that uses
Jinja2? Are there short-lived programs that use it? Python startup
time matters enormously to command-line tools like Mercurial, but far
less to something that's designed to start up and then keep running
(eg a web app, which is where Jinja is most used).

ChrisA