[Python-Dev] Is there a reference manual for Python bytecode?

[Ned Batchelder]

On 12/26/15 10:19 PM, Nick Coghlan wrote:
> On 27 December 2015 at 12:23, Guido van Rossum <guido at python.org> wrote:
>> On Sat, Dec 26, 2015 at 6:06 PM, Brett Cannon <brett at python.org> wrote:
>>> Ned also neglected to mention his byterun project which is a pure Python
>>> implementation of the CPython eval loop: https://github.com/nedbat/byterun
>>  From the commit log it looks like it's a co-production between Ned and
>> Allison Kaptur (who gave the talk I mentioned).
Yes, Allison was very helpful in pushing it forward.  And I should also 
mention that I started with a dormant project that Paul Swartz wrote.

And: it doesn't work completely.  There are things it doesn't handle 
properly, and I turned to other projects some time ago.  If someone 
wants to pick it up, that would be cool.
> It occurred to me that "byterun" would make a good see-also link from
> the dis module docs, and looking into that idea brought me to this
> article Allison wrote about it for the "500 lines" project:
> http://aosabook.org/en/500L/a-python-interpreter-written-in-python.html
>
> For a detailed semantic reference, byterun's eval loop is likely one
> of the most readable sources of information:
> https://github.com/nedbat/byterun/blob/master/byterun/pyvm2.py
I started working on byterun because I knew I didn't understand bytecode 
well enough to implement branch coverage measurement in coverage.py.  
Its primary goal was teaching (me!) how the bytecode worked.

Recently though, I've started a new implementation of branch coverage 
based on the ast rather than the bytecode.  This was prompted by the 
"async" keywords in 3.5.  "async for" and "for" compile very differently 
to bytecode, which caused headaches for a bytecode-based understanding 
of flow, so I'm trying out an ast-based understanding.

--Ned.
> In terms of formal documentation, the main problem with providing
> reference bytecode tables is keeping them up to date as the eval loop
> changes. However, it would theoretically be possible to create a
> custom Sphinx directive that uses the dis module to generate the
> tables automatically during the docs build process, rather than
> maintaining them by hand - something like that could be experimented
> with outside CPython, and potentially incorporated into the dis module
> docs if folks are able to figure out something that works well.
>
> Regards,
> Nick.
>