[pypy-issue] Issue #3105: Help in understanding code unmarshaling (pypy/pypy)

[Rocky Bernstein]

New issue 3105: Help in understanding code unmarshaling
https://bitbucket.org/pypy/pypy/issues/3105/help-in-understanding-code-unmarshaling

Rocky Bernstein:

I  am trying to get PyPy decompilation working on PyPy 7.1.1. Actually, though the problem is in unmarshaling code objects. 

I have tried to create a simple example which shows the problem, but alas, I am unable to do this. So the below is a little bit long in description, and I apologize for that.

.  

Lets first start out with this PyPy 3.6 byte compiled file fragment that has been has rendered. You can find the full file [here](https://github.com/rocky/python-uncompyle6/blob/pypy3.6/test/bytecode_pypy3.6/11_classbug.pyc).

```
$ uncompyle6 python-uncompyle6/test/bytecode_pypy3.6/11_classbug.pyc
00000000: a000 0d0a 8aa0 3257 8b00 0000 6300 0000  ......2W....c...
00000010: 0000 0000 0000 0000 0004 0000 0040 0000  ............. at ..
00000020: 0073 1400 0000 4700 6400 6401 8400 6401  .s....G.d.d...d.
00000030: 6500 8303 5a01 6402 5300 2903 6301 0000  e...Z.d.S.).c...
00000040: 0000 0000 0001 0000 0003 0000 0000 0000  ................
00000050: 0073 1c00 0000 6500 5a01 6401 5a02 8700  .s....e.Z.d.Z...
00000060: 6601 6402 6403 8408 5a03 8700 0400 5a04  f.d.d...Z.....Z.
00000070: 5300 2904 4ef4 1200 0000 5f54 656d 706c  S.).N....._Templ
00000080: 6174 654d 6574 6163 6c61 7373 6304 0000  ateMetaclassc...
00000090: 0000 0000 0004 0000 0005 0000 0003 0000  ................
000000a0: 0073 1800 0000 7400 7401 7c00 8302 c902  .s....t.t.|.....
000000b0: 7c01 7c02 7c03 ca03 0100 6400 5300 2901  |.|.|.....d.S.).
000000c0: 4e29 03f4 0500 0000 7375 7065 7272 0000  N)......superr..
000000d0: 0000 f408 0000 005f 5f69 6e69 745f 5f29  .......__init__)
000000e0: 04f4 0300 0000 636c 73f4 0400 0000 6e61  ......cls.....na
000000f0: 6d65 f505 0000 0062 6173 6573 f503 0000  me.....bases....
00000100: 0064 6374 2901 f409 0000 005f 5f63 6c61  .dct)......__cla
00000110: 7373 5f5f 2900 7520 0000 0073 696d 706c  ss__).u ...simpl
00000120: 655f 736f 7572 6365 2f64 6566 2f31 315f  e_source/def/11_
00000130: 636c 6173 7362 7567 2e70 7972 0200 0000  classbug.pyr....
                                     ^^ ^^
00000140: 0200 0000 7302 0000 0000 01f5 1b00 0000  ....s...........
```

I have underlined the important part of the code I will focus on below. 

Here is a fragment of a debug session of the cross-python disassembly called [  ](http://github.com/rocky/xdis)

‌

To compare against,  here is CPython 3.6 with corresponding but different bytecode, which can e found [ here](http://github.com/rocky/xdis). Below, inside a debugger ere the reference to `co_code` after `co_filename` is to object 3, and this is correct.

‌

```
00000100: 696d 706c 655f 736f 7572 6365 2f64 6566  imple_source/def
00000110: 2f31 315f 636c 6173 7362 7567 2e70 7972  /11_classbug.pyr
                                               ^^
00000120: 0300 0000 0200 0000 7302 0000 0000 017a  ........s......z
          ^^
00000130: 1b5f 5465 6d70 6c61 7465 4d65 7461 636c  ._TemplateMetacl
00000140: 6173 732e 5f5f 696e 6974 5f5f 2905 da08  ass.__init__)...
00000150: 5f5f 6e61 6d65 5f5f da0a 5f5f 6d6f 6475  __name__..__modu
00000160: 6c65 5f5f da0c 5f5f 7175 616c 6e61 6d65  le__..__qualname
00000170: 5f5f 7203 0000 00da 0d5f 5f63 6c61 7373  __r......__class
00000180: 6365 6c6c 5f5f 7208 0000 0072 0800 0000  cell__r....r....
00000190: 2901 7207 0000 0072 0900 0000 7201 0000  ).r....r....r...
000001a0: 0001 0000 0073 0200 0000 0801 7201 0000  .....s......r...
000001b0: 004e 2902 da04 7479 7065 7201 0000 0072  .N)...typer....r
000001c0: 0800 0000 7208 0000 0072 0800 0000 7209  ....r....r....r.
000001d0: 0000 00da 083c 6d6f 6475 6c65 3e01 0000  ........
000001e0:


(python-xdis/xdis/unmarshal.py:763 @20): t_object_reference
-- 763     o = internObjects[refnum - 1]
(trepan3k) refnum
3
(trepan3k) internObjects
['_TemplateMetaclass', 'super', '__init__', 'cls', 'name', 'bases', '__class__', (), 'simple_source/def/11_classbug.py']
(trepan3k) internObjects[refnum - 1]
'__init__'
(trepan3k) hex(fp.tell()-4)
'0x120'
```

‌

What's going on and how? Thanks!

  

‌

  What's going on and how can I fix? Thanks 

‌