Barry Warsaw writes:
A crazy offshoot idea would be something like Emacs' unexec, where during the build process you could preload a bunch of always-used immutable modules,
XEmacs doesn't do this any more if it can avoid it, we now have a portable dumper that we use on almost all platforms. And everybody at GNU Emacs who works with the unexec code wants to get rid of it. XEmacs's legacy unexec requires defeating address space randomization as well as certain optimizations that combine segments. I believe Emacs's does too. From a security standpoint, Emacsen are a child's garden of diseases and it will take decades, maybe centuries, to fix that, so those aren't huge problems for us. But I suppose Python needs to be able to work and play nicely with high-security environments, and would like to take advantage of security-oriented OS facilities like base address randomization. That kind of thing hugely complicates unexec -- last I heard it wasn't just "way too much work to be worth it", the wonks who created the portable dumper didn't know how to do it and weren't sure it could be done. XEmacs's default "portable dumper" is a poor man's relocating loader. I don't know exactly how it works, can't give details. Unlike the unexecs of some Lisps, however, this is a "do it once per build process" design. There's no explicit provision for keeping multiple dumpfiles around, although I believe it can be done "by hand" by someone with a little bit of knowledge. The reason for this is that the dumpfile is actually added to the executable. Regarding performance, the dumper itself is fast enough to be imperceptible to humans at load time, and doesn't take very long to build the dump file containing the "frozen" objects when building. I suspect Python has applications where it would be like to be faster than that, but I don't have benchmarks so don't know if this approach would be fast enough. This approach has the feature (disadvantage?) that some objects can't be dumped including editing buffers, network connections, and processes. I suppose those restrictions are very similar to the restrictions imposed by pickle. If somebody wants to know more about the portable dumper, I can probably connect them with the authors of that feature.