[Python-Dev] Python startup time

[Steve Dower]

“Stat calls in the import system were optimized in importlib a while back”

Yes, I’m aware of that, which is why I don’t have any specific suggestions off-hand. But given the differences in file systems between Windows and other OSs, it wouldn’t surprise me if there were a more optimal approach for NTFS to amortize calls better. Perhaps not, but it is still the most expensive part of startup that we have any ability to change, so it’s worth investigating.

Cheers,
Steve

Top-posted from my Windows phone

From: Brett Cannon
Sent: Saturday, July 22, 2017 10:18
To: Steve Dower; Alex Walters
Cc: Python-Dev
Subject: Re: [Python-Dev] Python startup time


On Sat, Jul 22, 2017, 07:22 Steve Dower, <steve.dower at python.org> wrote:
I believe the trend is due to language like Python and Node.js, most of which aggressively discourage threading (more from the broader community than the core languages, but I see a lot of apps using these now), and also the higher reliability afforded by out-of-process tasks (that is, one crash doesn’t kill the entire app – e.g browser tabs).
 
Optimizing startup time is incredibly valuable, and having tried it a few times I believe that the import system (in essence, stat calls) is the biggest culprit. The tens of ms prior to the first user import can’t really go anywhere.

Stat calls in the import system were optimized in importlib a while back to be cached in finders so at this point you will have to remove a stat call to lower that cost or cache more which goes into breaking abstractions or designing new APIs.

-brett

 
Cheers,
Steve
 
Top-posted from my Windows phone
 
From: Alex Walters
Sent: Saturday, July 22, 2017 1:39
Cc: 'Python-Dev'

Subject: Re: [Python-Dev] Python startup time
 
> -----Original Message-----
> From: Python-Dev [mailto:python-dev-bounces+tritium-
> list=sdamon.com at python.org] On Behalf Of Paul Moore
> Sent: Saturday, July 22, 2017 4:14 AM
> To: David Mertz <mertz at gnosis.cx>
> Cc: Barry Warsaw <barry at python.org>; Python-Dev <python-
> dev at python.org>
> Subject: Re: [Python-Dev] Python startup time
 
 
> It's a bit of a chicken and egg problem - Windows users avoid
> excessive command line program invocation because startup time is
> high, so no-one optimises startup time because Windows users don't use
> short-lived command line programs. But I'm seeing a trend away from
> that - more and more Windows tools these days seem to be comfortable
> spawning subprocesses. I don't know what prompted that trend.
 
The programs I see that are comfortable spawning processes willy-nilly on
windows are mostly .net, which has a lot of the runtime assemblies cached by
the OS in the GAC - if you are spawning a second processes of yourself, or
something that uses the same libraries as you, the compile step on those can
be skipped.  Unless you are talking about python/non-.NET programs, in which
case, I have no answer.
> Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium-
> list%40sdamon.com
 
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40python.org
 
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170722/a9d5a71f/attachment-0001.html>