On Fri, Apr 16, 2021 at 12:32 PM Larry Hastings <larry@hastings.org> wrote:

Please don't confuse Inada Naoki's benchmark results with the effect PEP 649 would have on a real-world codebase.  His artifical benchmark constructs a thousand empty functions that take three parameters with randomly-chosen annotations--the results provides some insights but are not directly applicable to reality.

PEP 649's effects on code size / memory / import time are contingent on the number of annotations and the number of objects annotated, not the overall code size of the module.  Expressing it that way, and suggesting that Python users would see the same results with real-world code, is highly misleading.

I too would be interested to know the effects PEP 649 had on a real-world codebase currently using PEP 563, but AFAIK nobody has reported such results.

I'm not going to report results, but we could use mypy itself as an example real-world code base. Mypy is almost 100% annotated. It does not include `from __future__ import annotations` lines but those could easily be added mechanically for some experiment.

ISTM that the unmarshal times reported by Inada are largely proportional to the code size numbers, so perhaps the following three-way experiment would give an indication:

(1) Addthe sizes of all pyc files for mypy run with Python 3.9 (classic)
(2) Ditto run with Python 3.10a7 (PEP 563)
(3) Ditto run with Larry's branch (PEP 649, assuming it's on by default there -- otherwise, modify the source by inserting the needed future import at the top)

The repo is github.com/python/mypy, the subdirectory to look is mypy, WITH THE EXCLUSION OF THE typeshed SUBDIRECTORY THEREOF.

--Guido van Rossum (python.org/~guido)