[Python-Dev] Cython compiled stdlib modules - Re: Python startup time

[Nick Coghlan]

On 22 July 2017 at 06:43, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Nick Coghlan schrieb am 21.07.2017 um 08:23:
>> I'll also note that in these cases where the import overhead is
>> proportionally significant for always-imported modules, we may want to look
>> at the benefits of freezing them (if they otherwise remain as pure Python
>> modules), or compiling them as builtin modules (if we switch them over to
>> Cython), in addition to looking at ways to make the modules themselves
>> faster.
>
> Just for the sake of it, I gave the Cython compilation a try. I had to
> apply the attached hack to Lib/typing.py to get the test passing, because
> it uses frame call offsets in some places and Cython functions do not
> create frames when being called (they only create them for exception
> traces). I also had to disable the import of "abc" in the Cython generated
> module to remove the circular self dependency at startup when the "abc"
> module is compiled. That shouldn't have an impact on the runtime
> performance, though.

[snip]

> As it stands, the gain is probably not worth the increase in library file
> size, which also translates to a higher bottom line for the memory
> consumption. At least not for these two modules. Manually optimising the
> files would likely also reduce the .so file size in addition to giving
> better speedups, though, because the generated code would become less generic.

Thanks for trying the experiment! I agree with your conclusion that
the file size impact likely rules it out as a general technique.

Selective freezing may still be interesting though, since that at
least avoids the import path searches and merges the disk read into
the initial loading of the executable.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia