[Cython] Cython compilation speed vs. installed binary module sizes

[Stefan Behnel]

Hi all,

when building Cython, it compiles a couple of its own modules. I was
interested in what this brings for the separate parts of the compiler, so I
used the compilation of Cython's compiled modules as benchmark and compiled
the separate parts incrementally, using the latest CPython 3.7.0.

Baseline, uncompiled Python run:

real    0m14.972s
user    0m14.565s
sys     0m0.165s

Compiling the parser:
Cython/Compiler/Scanning.py  Cython/Plex/Scanners.py
Cython/Plex/Actions.py  Cython/Compiler/Parsing.py

real    0m13.614s
user    0m13.320s
sys     0m0.136s

Additionally:
Cython/Compiler/Visitor.py  Cython/Compiler/FlowControl.py

real    0m7.857s
user    0m7.778s
sys     0m0.074s

Additionally:
Cython/Compiler/Code.py  Cython/StringIOTree.py

real    0m7.356s
user    0m7.068s
sys     0m0.102s


Only compiling the tree visitor and the control flow analysis:
Cython/Compiler/Visitor.py  Cython/Compiler/FlowControl.py

real    0m9.029s
user    0m8.899s
sys     0m0.082s


These are the current shared library sizes on my machine:

  57040 Cython/Plex/Actions.cpython-37m-x86_64-linux-gnu.so
  72592 Cython/StringIOTree.cpython-37m-x86_64-linux-gnu.so
  78880 Cython/Runtime/refnanny.cpython-37m-x86_64-linux-gnu.so
  86384 Cython/Plex/Scanners.cpython-37m-x86_64-linux-gnu.so
 127952 Cython/Compiler/Pythran.cpython-37m-x86_64-linux-gnu.so
 227480 Cython/Compiler/Scanning.cpython-37m-x86_64-linux-gnu.so
 322136 Cython/Compiler/Visitor.cpython-37m-x86_64-linux-gnu.so
 559864 Cython/Tempita/_tempita.cpython-37m-x86_64-linux-gnu.so
 598712 Cython/Compiler/FlowControl.cpython-37m-x86_64-linux-gnu.so
 902904 Cython/Compiler/Parsing.cpython-37m-x86_64-linux-gnu.so
1133176 Cython/Compiler/Code.cpython-37m-x86_64-linux-gnu.so
4167120 insgesamt


This suggests that by only compiling the 6 modules of the scanner, parser,
tree visitor and CFA, we get about half the binary sizes, but almost the
same speed.

One caveat: Being pure Python, Cython does not use all that many of its own
language features, so other code might suffer more or less. I also tried
lxml's main module, and it compiles about 7% slower when removing Code.so.
Given that it's the biggest of the modules, that feels acceptable for me,
so I removed Code, Pythran, Lexicon and StringIOTree from the list of
compiled modules for now.

Stefan