On Sat, May 5, 2018, 11:34 Toshio Kuratomi <a.badger@gmail.com> wrote:


On Sat, May 5, 2018, 10:40 AM Eric Fahlgren <ericfahlgren@gmail.com> wrote:
On Sat, May 5, 2018 at 10:30 AM, Toshio Kuratomi <a.badger@gmail.com> wrote:
On Fri, May 4, 2018, 7:00 PM Nathaniel Smith <njs@pobox.com> wrote:
What are the obstacles to including "preloaded" objects in regular .pyc files, so that everyone can take advantage of this without rebuilding the interpreter?

Would this make .pyc files arch specific?

Or have parallel "pyh" (Python "heap") files, that are architecture specific... (But that would cost more stat calls.)

I ask because arch specific byte code files are a big change in consumers expectations.  It's not necessarily a bad change but it should be communicated to downstreams so they can decide how to adjust to it.  

Linux distros which ship byte code files will need to build them for each arch, for instance.  People who ship just the byte code as an obfuscation of the source code will need to decide whether to ship packages for each arch they care about or change how they distribute.

That's a good point.

One way to minimize the disruption would be to include both the old and new info in the .pyc files, so at load time if the new version is incompatible then you can fall back on the old way, even if it's a bit slower.

I think in the vast majority of cases currently .pyc files are built on the same architecture where they're used? Pip and Debian/Ubuntu and the interpreter's automatic compilation-on-import all build .pyc files on the computer where they'll be run.

It might also be worth double checking much the memory layout of these objects even varies. Obviously it'll be different for 32- and 64-bit systems, but beyond that, most ISAs and OSes and compilers use pretty similar struct layout rules AFAIK... we're not talking about actual machine code.

-n