On Fri, Jan 20, 2017 at 8:17 PM, Victor Stinner
2017-01-20 11:49 GMT+01:00 INADA Naoki
: Report is here https://gist.github.com/methane/ce723adb9a4d32d32dc7525b738d3c31
Very interesting report, thanks!
My thoughts are:
* Interning (None,) seems worth enough.
I guess that (None,) comes from constants of code objects:
def f(): pass ... f.__code__.co_consts (None,)
* There are many empty dicts. Allocating ma_keys lazily may reduce memory usage.
Would you be able to estimate how many bytes would be saved by this change? With the total memory usage to have an idea of the %.
Smallest dictkeysobject is 5*8 + 8 + (8 * 3 * 5) = 168 bytes. 1600 empty dicts = 268800 bytes. Unlike tuples bound to code objects, I don't think this is so important for cache hit rate. So tuple is more important than dict.
By the way, it would help if you can add the total memory usage computed by tracemalloc (get_traced_memory()[0]) in your report.
Oh, nice to know it. I'll use it next time.
About empty dict, do you expect that they come from shared keys? Anyway, if it has a negligible impact on the performance, go for it :-)
but other namespaces or annotations, like ('return',) or ('__wrapped__',) are not shared
Maybe we can intern all tuple which only contains one string?
Ah, it's dict's key. I used print(tuple(d.keys())) to count dicts.
Instead of interning, would it be possible to at least merge duplicated immutable objects?
I meant sharing same object, I didn't meant using dict or adding bit for interning like interned strings. So I think we have same idea.
* Most large strings are docstring. Is it worth enough that option for trim docstrings, without disabling asserts?
Yeah, you are not the first one to propose. The problem is to decide how to name the .pyc file.
My PEP 511 proposes to add a new -O command line option and a new sys.implementation.optim_tag string to support this feature: https://www.python.org/dev/peps/pep-0511/
Since the code transformer part of the PEP seems to be controversal, maybe we should extract only these two changes from the PEP and implement them? I also want -O noopt :-) (disable the peephole optimizer)
Victor