[Python-Dev] Python startup optimization: script vs. service

[Nick Coghlan]

On 3 October 2017 at 03:02, Christian Heimes <christian at python.org> wrote:
> On 2017-10-02 16:59, Barry Warsaw wrote:
>> On Oct 2, 2017, at 10:48, Christian Heimes <christian at python.org> wrote:
>>>
>>> That approach could work, but I think that it is the wrong approach. I'd
>>> rather keep Python optimized for long-running processes and introduce a
>>> new mode / option to optimize for short-running scripts.
>>
>> What would that look like, how would it be invoked, and how would that change the behavior of the interpreter?
>
> I haven't given it much thought yet. Here are just some wild ideas:
>
> - add '-l' command line option (l for lazy)
> - in lazy mode, delay some slow operations (re compile, enum, ...)
> - delay some imports in lazy mode, e.g. with a deferred import proxy

I don't think is the right way to structure the defaults, since the
web services world is in the middle of moving back closer to the
CLI/CGI model, where a platform like AWS Lambda will take care of
spinning up language interpreter instances on demand, using them to
process a single request, and then discarding them.

It's also somewhat unreliable to pass command line options as part of
shebang lines, and packaging tools need to be able to generate shebang
lines that are compatible with a wide variety of Python
implementations

By contrast, long running Python services will typically be using some
form of WSGI server (whether that's mod_wsgi, uWSGI, gunicorn,
Twisted, tornado, or something else) that can choose to adjust *their*
defaults to force the underlying language runtime into an "eager state
initialisation" mode, even if the default setting is to respect
requests for lazy initialisation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia