[Python-Dev] Intention to accept PEP 552 soon (deterministic pyc files)

[Nick Coghlan]

On 5 October 2017 at 23:44, Barry Warsaw <barry at python.org> wrote:

> On Oct 4, 2017, at 13:53, Benjamin Peterson <benjamin at python.org> wrote:
>
> > It might be helpful to enumerate the usecases for such an API. Perhaps a
> > narrow, specialized API could satisfy most needs in a supportable way.
>
> Currently `python -m dis thing.py` compiles the source then disassembles
> it.  It would be kind of cool if you could pass a .pyc file to -m dis, in
> which case you’d need to unpack the header to get to the code object.  A
> naive implementation would unpack the magic number and refuse to
> disassemble any files that don’t match whatever that version of Python
> understands.  A more robust (possibly 3rd party) implementation could
> potentially disassemble a range of magic numbers and formats, and an API to
> get at the code object and metadata would help.
>
> I was thinking about the bytecode hacking that some debuggers do.  This
> API would help them support multiple versions of Python.  They could use
> the API to discover what pyc format was in use, extract the code object,
> hack the bytecode and possibly rewrite a new PEP 3147 style pyc file with
> the debugger bytecodes inserted.
>
> Third party bytecode optimizers could use the API to unpack multiple
> versions of pyc files, do their optimizations, and rewrite new files with
> the proper format.
>

Actually doing that properly also requires keeping track of which opcodes
were valid in different versions of the eval loop, so as Guido suggests,
such an abstraction layer would make the most sense as a third party
project that tracked:

- the magic number for each CPython feature release (plus the 3.5.3+
anomaly)
- the pyc header format for each CPython feature release
- the valid opcode set for each CPython feature release
- any other version dependent variations (e.g. the expected stack layout
for BUILD_MAP changed in Python 3.5, when the evaluation order for dict
displays was updated to be key then value, rather than the other way around)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171006/f69d6ca4/attachment-0001.html>