[Python-Dev] Cython compiled stdlib modules - Re: Python startup time

[Stefan Behnel]

Nick Coghlan schrieb am 21.07.2017 um 08:23:
> I'll also note that in these cases where the import overhead is
> proportionally significant for always-imported modules, we may want to look
> at the benefits of freezing them (if they otherwise remain as pure Python
> modules), or compiling them as builtin modules (if we switch them over to
> Cython), in addition to looking at ways to make the modules themselves
> faster.

Just for the sake of it, I gave the Cython compilation a try. I had to
apply the attached hack to Lib/typing.py to get the test passing, because
it uses frame call offsets in some places and Cython functions do not
create frames when being called (they only create them for exception
traces). I also had to disable the import of "abc" in the Cython generated
module to remove the circular self dependency at startup when the "abc"
module is compiled. That shouldn't have an impact on the runtime
performance, though.

Note that this is otherwise using the unmodified Python code, as provided
in the current modules, constructing and using normal Python classes for
everything, no extension types etc. Only two stdlib Python modules were
compiled into shared libraries, and not statically linked into the CPython
core.


I used the "python_startup" benchmark in the "performance" project to
measure the overall startup times of a clean non-debug non-pgo build of
CPython 3.7 (rev d0969d6) against the same build with a compiled typing.py
and abc.py. To compile these modules, I used the following command (plus
the attached patch)

$ cythonize -3 -X binding=True -i Lib/typing.py Lib/abc.py

I modified the startup benchmark to run "python -c 'import typing'" etc.
instead of just executing "pass".


- stock CPython starting up and running "pass":
Mean +- std dev: 14.7 ms +- 0.3 ms


- stock CPython starting up and running "import abc":
Mean +- std dev: 14.8 ms +- 0.3 ms

- with compiled abc.py:
Mean +- std dev: 14.9 ms +- 0.3 ms


- stock CPython starting up and running "import typing":
Mean +- std dev: 34.6 ms +- 1.0 ms

- with compiled abc.py
Mean +- std dev: 34.4 ms +- 0.6 ms

- with compiled typing.py:
Mean +- std dev: 33.5 ms +- 0.7 ms

- with both compiled:
Mean +- std dev: 33.1 ms +- 0.4 ms

That's only a 4% improvement in the overall startup time on my machine, and
about a 7% faster overall runtime of "import typing" compared to "pass".
Note also that compiling abc.py leads to a slightly *increased* startup
time in the "import abc" case, which might be due to the larger file size
of the abc.so file compared to the abc.pyc file. This is amortised by the
decreased runtime in the "import typing" case (I guess).


I then ran the test suites for both modules in lack of a better
post-startup runtime benchmark. The improvement for abc.py is in the order
of 1-2%, but test_typing.py has many more tests and wins about 13% overall:

- stock CPython executing essentially "runner.run(deepcopy(suite))" in
"test_typing.py" (the deepcopy() takes about 6 ms):
Mean +- std dev: 68.6 ms +- 0.8 ms

- compiled abc.py and typing.py:
Mean +- std dev: 60.7 ms +- 0.7 ms


One more thing to note: the compiled modules are quite large. I get these
file sizes:

   8658  Lib/abc.py
   7525  Lib/__pycache__/abc.cpython-37.pyc
 369930  Lib/abc.c
 122048  Lib/abc.cpython-37m-x86_64-linux-gnu.so

  80290  Lib/typing.py
  73921  Lib/__pycache__/typing.cpython-37.pyc
2951893  Lib/typing.c
1182632  Lib/typing.cpython-37m-x86_64-linux-gnu.so

The .so files are about 16x as large as the .pyc files. The typing.so file
weighs in with about 40% of the size of the stripped python binary:

2889136  python


As it stands, the gain is probably not worth the increase in library file
size, which also translates to a higher bottom line for the memory
consumption. At least not for these two modules. Manually optimising the
files would likely also reduce the .so file size in addition to giving
better speedups, though, because the generated code would become less generic.

Stefan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: typing_frames.patch
Type: text/x-patch
Size: 1439 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20170721/79022bb8/attachment.bin>