[Python-Dev] Lazy initialization of module global state
neil at python.ca
Fri Sep 8 11:50:12 EDT 2017
This is an idea that came out of the lazy module loading (via AST
analysis), posted to python-ideas. The essential idea is to split
the marshal data stored in the .pyc into smaller pieces and only
load the parts as they are accessed. E.g. use a __getattr__ hook on
the module to unmarshal+exec the code.
I have a very early prototype:
Would work like a "compile_all.py" tool. It writes standard .pyc
files right now. It is not there yet but I should use the AST
analysis, like the lazy module load stuff, to determine if things
have potential side-effects on module import. Those things will get
loaded eagerly, like they do now.
Initially I was thinking of class definitions and functions but now
I realize any global state could get this treatment. E.g. if you
have a large dictionary global, don't unmarshal it until someone
accesses the module attribute.
This should be pretty safe to do and should give a significant
benefit in startup time and memory usage.
More information about the Python-Dev