On 5 May 2018 at 11:58, Nathaniel Smith <njs@pobox.com> wrote:
What are the obstacles to including "preloaded" objects in regular .pyc files, so that everyone can take advantage of this without rebuilding the interpreter?

Off the top of my head:

We'd be making the in-memory layout of those objects part of the .pyc format, so we couldn't change that within a minor release. I suspect this wouldn't be a big change though, since we already commit to ABI compatibility for C extensions within a minor release? In principle there are some cases where this would be different (e.g. adding new fields at the end of an object is generally ABI compatible), but this might not be an issue for the types of objects we're talking about.

I'd frame this one a bit differently: what if we had a platform-specific variant of the pyc format that was essentially a frozen module packaged as an extension module? We probably couldn't quite do that for arbitrary Python modules *today* (due to the remaining capability differences between regular modules and extension modules), but multi-phase initialisation gets things *much* closer to parity, and running embedded bytecode instead of accessing the C API directly should avoid the limitations that exist for classes defined in C.


Nick Coghlan   |   ncoghlan@gmail.com   |   Brisbane, Australia