[Python-Dev] To reduce Python "application" startup time

[Barry Warsaw]

On Sep 6, 2017, at 00:42, INADA Naoki <songofacandy at gmail.com> wrote:

> Additionally, faster startup time (and smaller memory footprint) is good
> for even Web applications.
> For example, CGI is still comfortable tool sometimes.
> Another example is GAE/Python.
> 
> Anyway, I think researching import tree of popular library is good startline
> about optimizing startup time.
> For example, modules like ast and tokenize are imported often than I thought.

Improving start up time may indeed help long running processes but start up costs will generally be amortized across the lifetime of the process, so it isn’t as noticeable.  However, startup time *is* a real issue for command line tools.

I’m not sure however whether burying imports inside functions (as a kind of poor man’s lazy import) is ultimately going to be satisfying.  First, it’s not natural, it generally violates coding standards (e.g. PEP 8), and can make linters complain.  Second, I think you’ll end up chasing imports all over the stdlib and third party modules in any sufficiently complicated application.  Third, I’m not sure that the gains you’ll get won’t just be overwhelmed by lots of other things going on, such as pkg_resources entry point processing, pth file processing, site.py effects, command line processing libraries such as click, and implicitly added distribution exception hooks (e.g. Ubuntu’s apport).

Many of these can’t be blamed on Python itself, but all can contribute significantly to Python’s apparent start up time.  It’s definitely worth investigating the details of Python import, and a few of us at the core sprint have looked at those numbers and thrown around ideas for improvement, but we’ll need to look at the effects up and down the stack to improve the start up performance for the average Python application.

Cheers,
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 273 bytes
Desc: Message signed with OpenPGP
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170906/05643dac/attachment.sig>