I updated the benchmark little: * Added no annotation mode for baseline performance. * Better stats output. https://gist.github.com/methane/abb509e5f781cc4a103cc450e1e7925d ``` # No annotation (master + GH-25419) $ ./python ~/ann_test.py 0 code size: 102967 bytes memory: 181288 bytes unmarshal: avg: 299.301ms +/-1.257ms exec: avg: 104.019ms +/-0.038ms # PEP 563 (master + GH-25419) $ ./python ~/ann_test.py 2 code size: 110488 bytes memory: 193572 bytes unmarshal: avg: 313.032ms +/-0.068ms exec: avg: 108.456ms +/-0.048ms # PEP 649 (co_annotations + GH-25419 + GH-23056) $ ./python ~/ann_test.py 3 code size: 204963 bytes memory: 209257 bytes unmarshal: avg: 587.336ms +/-2.073ms exec: avg: 97.056ms +/-0.046ms # Python 3.9 $ python3 ann_test.py 0 code size: 108955 bytes memory: 173296 bytes unmarshal: avg: 333.527ms +/-1.750ms exec: avg: 90.810ms +/-0.347ms $ python3 ann_test.py 1 code size: 121011 bytes memory: 385200 bytes unmarshal: avg: 334.929ms +/-0.055ms exec: avg: 400.260ms +/-0.249ms ``` ## Rough estimation of annotation overhead Python 3.9 w/o PEP 563 code (pyc) size: +11% memory usage: +122% (211bytes / function) import time: +73% (*) PEP 563 code (pyc) size: +7.3% memory usage: +0.68% (13.3bytes / function) import time: +4.5% PEP 649 code (pyc) size: +99% memory usage: +15% (28 bytes / function) import time: +70% (*) import time can be much more slower for complex annotations. ## Conclusion * PEP 563 is close to "zero overhead" in memory consumption. And import time overhead is ~5%. Users can write type annotations without caring overhead. * PEP 649 is much better than old semantics for memory usage and import time. But import time is still longer than no annotation code. * The import time overhead is coming from unmarshal, not from eval(). If we implement a "lazy load" mechanizm for docstrings and annotations, overhead will become cheaper. * pyc file become bigger (but who cares?) I will read PEP 649 implementation to find missing optimizations other than GH-25419 and GH-23056. -- Inada Naoki <songofacandy@gmail.com>