[Python-Dev] A fast startup patch (was: Python startup time)

[Terry Reedy]

On 5/5/2018 2:33 PM, Toshio Kuratomi wrote:
> 
> 
> On Sat, May 5, 2018, 10:40 AM Eric Fahlgren <ericfahlgren at gmail.com 
> <mailto:ericfahlgren at gmail.com>> wrote:
> 
>     On Sat, May 5, 2018 at 10:30 AM, Toshio Kuratomi <a.badger at gmail.com
>     <mailto:a.badger at gmail.com>>wrote:
> 
>         On Fri, May 4, 2018, 7:00 PM Nathaniel Smith <njs at pobox.com
>         <mailto:njs at pobox.com>> wrote:
> 
>             What are the obstacles to including "preloaded" objects in
>             regular .pyc files, so that everyone can take advantage of
>             this without rebuilding the interpreter?
> 
> 
>         Would this make .pyc files arch specific?
> 
> 
>     Or have parallel "pyh" (Python "heap") files, that are architecture
>     specific... (But that would cost more stat calls.)
> 
> 
> I ask because arch specific byte code files are a big change in 
> consumers expectations.  It's not necessarily a bad change but it should 
> be communicated to downstreams so they can decide how to adjust to it.
> 
> Linux distros which ship byte code files will need to build them for 
> each arch, for instance.  People who ship just the byte code as an 
> obfuscation of the source code will need to decide whether to ship 
> packages for each arch they care about or change how they distribute.

It is an advertised feature that CPython *can* produce cross-platform 
version-specific .pyc files.  I believe this should continue, at least 
for a few releases.  They are currently named modname.cpython-xy.pyc, 
with optional '.opt-1', '.opt-2', and '.opt-4' tags inserted before  in 
__pycache__.  These name formats should continue to mean what they do now.

I believe *can* should not mean *always*.  Architecture-specific files 
will need an additional architecture tag anyway, such as win32 and 
win64, anyway.  Or would bitness and endianess be sufficient across 
platforms?   If we make architecture-specific the default, we could add 
startup and compile and compile_all options for the cross-platform 
format.  Or maybe add a recompile function that imports cross-platform 
.pycs and outputs local-architecture .pycs.

-- 
Terry Jan Reedy