I encourage you to search this list (and maybe Python-dev) for a discussion (maybe a year ago or so?) about speeding up interpreter startup. In particular, there were ideas about having a bi art pre-loaded with all the standard modules that get loaded every time. The conversation petered out, but there were some good ideas on the table that could be further explored. -CHB On Mon, Sep 16, 2019 at 4:20 AM Anders Hovmöller wrote:

Hi,

I would like some feedback on the feasibility of an idea I've had: caching an entire import graph to speed up interactive apps. The idea is that tools like pytest, pip, hg, etc would be able to save and restore an import graph to cut down on initial startup time.

As I think of it, this will be an opt-in system where a cache file could be stored on disk and loaded quickly and then gone through with one or two passes to relocate pointers to the present values if needed. The details of the API to be determined later. I'm thinking it would require changes to pip/setuptools to clear these caches when a package is upgraded for example.

I have very limited experience with the CPython code base so I'd like to know if this is even a feasible thing to do?

Best regards Anders _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VPHG4Q... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

Wow! That's pretty great. I'm watching this tomorrow for sure. Do you have a feeling on if your patch is scalable up from your use case of the basic standard modules all the way up to pytest?

On 16 Sep 2019, at 18:43, jeethu wrote:

I'd hacked together an implementation of this idea last year for Python 3.6 and even gave a talk[1] on it at last year's EuroPython.

Larry Hastings was interested in this idea, so I'd sent him my patch. He seems to have ported it to Python 3.8 and it's on this issue[2] on the bug tracker.

- Jeethu

[1]: https://www.youtube.com/watch?v=KRqv2Bm1J18 [2]: https://bugs.python.org/issue34690

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

...

On Monday, September 16, 2019 12:19 PM, Anders Hovmöller wrote:

Hi,

I would like some feedback on the feasibility of an idea I've had: caching an entire import graph to speed up interactive apps. The idea is that tools like pytest, pip, hg, etc would be able to save and restore an import graph to cut down on initial startup time.

As I think of it, this will be an opt-in system where a cache file could be stored on disk and loaded quickly and then gone through with one or two passes to relocate pointers to the present values if needed. The details of the API to be determined later. I'm thinking it would require changes to pip/setuptools to clear these caches when a package is upgraded for example.

I have very limited experience with the CPython code base so I'd like to know if this is even a feasible thing to do?

Best regards Anders

Hi I'm interested in this area. Jeethu's slides are available at https://ep2018.europython.eu/media/conference/slides/faster-python-startup.p... By the way, TeX has a \dump command, which also makes it an example of an "image-based language", as in Jeethu's slides. LaTeX is an example of a \dump used to make an executable (together with a run-time library). https://tug.org/texinfohtml/web2c.html#Memory-dumps best regards Jonathan