[Python-ideas] More compact bytecode

[Andrew Barnert]

On Feb 4, 2016, at 10:55, Serhiy Storchaka <storchaka at gmail.com> wrote:

>> On 04.02.16 19:27, Sven R. Kunze wrote:
>> On 04.02.2016 17:55, Serhiy Storchaka wrote:
>>>>> 1. Add 1-byte dedicated opcodes for most used opcodes with arguments.
>> 
>> Do you have a working CPython to showcast this?
> 
> Not yet. The interpreter is easy, but the compiler is more complicated in this case.
> 
>>>>> 2. Use 16-bit opcodes as in WPython.
>> 
>> Same question as above + does it make sense to combine those two options?
> 
> Of course not. Having some opcodes to be 8-bit kills the benefit of all 16-bit opcodes.
> 

> This options needs to rewrite more code, but resulting code can be easier. I afraid that might be problem with bootstrapping, generating freezed modules.
I just tried it out to see what's hard. And it seems like it's all easy. I was able to get it working (not passing all tests, of course--you'd have to fix dis, pdb, etc. for that) on my lunch hour.


I didn't try to be smart anywhere: every opcode gets 8 bits for an arg, and uses up to three EXTENDED_ARG ops if it needs more. I didn't do any simplification, nor did I cast the bytecode to unsigned short *.
The hardest changes are in ceval (plus frameobject--I didn't realize lasti has to start lasti at -2 instead of -1 until I tried to run it and immediately got errors about unknown opcodes at odd offsets).

Compile is pretty easy by comparison.

Nothing at all has to be done for bootstrap, freeze, marshal, etc. (unless you want to add backward-compat code to load non-word-based .pyc files).

You'd also need to fix at least peephole (I just disabled it), dis, and pdb, but none of them look too hard.

The bootstrap bytecode and stdlib .pyc files ends up about 5% smaller (and that's with the peephole optimizer disabled, so presumably it would be better in real life).


If anyone wants to look at or play with the code, I'm in the process of regenerating my github fork, but it'll be branch wpy on https://github.com/abarnert/cpython once that's done (maybe 20 minutes? I forget how long cloning takes).