Mailman 3 November 2021 - pypy-dev

PyPy Winter Sprint + cpyext
by Stefan Beyer 21 Dec '22

21 Dec '22

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

How to fix pypy3 runtime error ?
by Hao Wang 29 Nov '21

29 Nov '21

Dear devs: I encountered the following error but didn't know how to fix it. Please help. cpyext, the emulation layer, detected that while it is calling an object's tp_dealloc, the C code calls back a function that tries to recreate the PyPy version of the object. Usually it means that tp_dealloc calls some general PyXxx() API. It is a dangerous and potentially buggy thing to do: even in CPython the PyXxx() function could, in theory, cause a reference to the object to be taken and stored somewhere, for an amount of time exceeding tp_dealloc itself. Afterwards, the object will be freed, making that reference point to garbage. >>> PyPy could contain some workaround to still work if you are lucky, but it is not done so far; better fix the bug in the CPython extension. >>> This object is of type 'pandas._libs.parsers.TextReader' Aborted Bravo ! Hao Wang

2 2

pip installation for pypy is slow
by Hao Wang 29 Nov '21

29 Nov '21

Dear pypy-devs, The pip installation of pypy (pypy -m pip install scipy, for example) is slower than python-pip clean. Is there a way to fix this problem ? Bravo! Hao Wang

2 2

Re: [pypy-dev] How to compile a file with rpython
by M A 22 Nov '21

22 Nov '21

> On Nov 22, 2021, at 1:08 AM, Dan Stromberg <drsalists(a)gmail.com> wrote: > > > On Sun, Nov 21, 2021 at 11:17 AM M A <teammember0x01(a)gmail.com> wrote: > How do I make a program using rpython? > > Say I have this file: > > def main(): > for i in range(100): > print("hello world"), > > if __name__ == "__main__": > main() > > How would I turn it into an executable file using rpython? > > Why do you desire an executable? > > Last I heard, a program would perform well on Pypy's JIT, and that using rpython was almost always unnecessary. > > If you need an executable, you could also try Cython or Nuitka. Well I'm working on code for PyPy. If I could unit test my code my development speed would increase a lot. It takes a long time for PyPy to build on my computer.

2 1

How to compile a file with rpython
by M A 22 Nov '21

22 Nov '21

How do I make a program using rpython? Say I have this file: def main(): for i in range(100): print("hello world"), if __name__ == "__main__": main() How would I turn it into an executable file using rpython?

2 1

Identify this function
by M A 15 Nov '21

15 Nov '21

for i in range(10000): print(i) Hi, when I run this loop PyPy crashes at around 8800. Using a debugger I found out that this is the AARCH64 assembly function that has the crash. The arrow points to where the crash happens. I believe the crash happens one instruction before the arrow, at blr x17. X17 was set to a small number that was not a valid address. Would anyone know what the rpython version of this function is? Thank you. 0x10a97cf10: str x0, [x29, #0x48] 0x10a97cf14: str x1, [x29, #0x50] 0x10a97cf18: str x2, [x29, #0x58] 0x10a97cf1c: str x3, [x29, #0x60] 0x10a97cf20: str x4, [x29, #0x68] 0x10a97cf24: str x5, [x29, #0x70] 0x10a97cf28: str x6, [x29, #0x78] 0x10a97cf2c: str x7, [x29, #0x80] 0x10a97cf30: str x8, [x29, #0x88] 0x10a97cf34: str x9, [x29, #0x90] 0x10a97cf38: str x10, [x29, #0x98] 0x10a97cf3c: str x11, [x29, #0xa0] 0x10a97cf40: str x12, [x29, #0xa8] 0x10a97cf44: str x13, [x29, #0xb0] 0x10a97cf48: str x19, [x29, #0xb8] 0x10a97cf4c: str x20, [x29, #0xc0] 0x10a97cf50: str d0, [x29, #0xc8] 0x10a97cf54: str d1, [x29, #0xd0] 0x10a97cf58: str d2, [x29, #0xd8] 0x10a97cf5c: str d3, [x29, #0xe0] 0x10a97cf60: str d4, [x29, #0xe8] 0x10a97cf64: str d5, [x29, #0xf0] 0x10a97cf68: str d6, [x29, #0xf8] 0x10a97cf6c: str d7, [x29, #0x100] 0x10a97cf70: sub sp, sp, #0x10 ; =0x10 0x10a97cf74: str x16, [sp, #0x8] 0x10a97cf78: str x30, [sp] 0x10a97cf7c: blr x17 -> 0x10a97cf80: mov x16, #0x3428 0x10a97cf84: movk x16, #0x69f, lsl #16 0x10a97cf88: movk x16, #0x1, lsl #32 0x10a97cf8c: ldr x16, [x16] 0x10a97cf90: sub x16, x16, #0x8 ; =0x8 0x10a97cf94: ldr x29, [x16] 0x10a97cf98: ldrb w16, [x29, #0x4] 0x10a97cf9c: mov x17, #0x1 0x10a97cfa0: tst x16, x17 0x10a97cfa4: b.eq 0x10a97cfd8 0x10a97cfa8: sub sp, sp, #0x10 ; =0x10 0x10a97cfac: str x0, [sp, #0x8] 0x10a97cfb0: str x29, [sp] 0x10a97cfb4: mov x0, x29 0x10a97cfb8: mov x16, #0xc290 0x10a97cfbc: movk x16, #0xa97, lsl #16 0x10a97cfc0: movk x16, #0x1, lsl #32 0x10a97cfc4: movk x16, #0x0, lsl #48 0x10a97cfc8: blr x16 0x10a97cfcc: ldr x0, [sp, #0x8] 0x10a97cfd0: ldr x29, [sp] 0x10a97cfd4: add sp, sp, #0x10 ; =0x10 0x10a97cfd8: mov x17, x0 0x10a97cfdc: ldr x0, [x29, #0x48] 0x10a97cfe0: ldr x1, [x29, #0x50] 0x10a97cfe4: ldr x2, [x29, #0x58] 0x10a97cfe8: ldr x3, [x29, #0x60] 0x10a97cfec: ldr x4, [x29, #0x68] 0x10a97cff0: ldr x5, [x29, #0x70] 0x10a97cff4: ldr x6, [x29, #0x78] 0x10a97cff8: ldr x7, [x29, #0x80] 0x10a97cffc: ldr x8, [x29, #0x88] 0x10a97d000: ldr x9, [x29, #0x90] 0x10a97d004: ldr x10, [x29, #0x98] 0x10a97d008: ldr x11, [x29, #0xa0] 0x10a97d00c: ldr x12, [x29, #0xa8] 0x10a97d010: ldr x13, [x29, #0xb0] 0x10a97d014: ldr x19, [x29, #0xb8] 0x10a97d018: ldr x20, [x29, #0xc0] 0x10a97d01c: ldr x16, [sp] 0x10a97d020: add sp, sp, #0x10 ; =0x10 0x10a97d024: ret x16

1 0

Debugging symbols for JIT compiled code?
by M A 15 Nov '21

15 Nov '21

Is it possible for PyPy to add debugging symbols to the code it creates and writes to memory? I'm trying to figure out why my python program keeps crashing. If I had debugging symbols available I could instantly figure out what function is being called. Thank you.

1 0

macos buildbot offline for a few days
by Matt Billenstein 12 Nov '21

12 Nov '21

Starting tomorrow morning PST, I'm moving, so it should be back sometime Saturday evening or perhaps Monday if I have trouble with the internet self-install. thx m -- Matt Billenstein matt(a)vazor.com http://www.vazor.com/

2 1

plans for the peg parser in pypy 3.9
by Carl Friedrich Bolz-Tereick 10 Nov '21

10 Nov '21

Hi all, so I've been hacking on getting an rpython version of the new peg parser that cpython introduced in 3.9, and switched exclusively to in 3.10. My status on the py3.9-peg-parser branch is that I managed to build a full pypy-c with the new parser for the first time today, but there are still a number of bugs (in particular, the interactive console is pretty broken). I wanted to keep you in the loop on my plans for further steps: - I plan to fix the bugs, obviously ;-) - CPython made it possible to switch between the old and the new parser in 3.9 with an environment variable. Unless somebody gives extremely strong reasons for also having that feature, I would like to focus on using the new parser exclusively. Being able to switch is quite a big hassle, and I don't see it buying a lot. - That said, I intend to keep the "parser" module functional for 3.9, like CPython did. It will be removed (for us and for them) in 3.10. Let me know if you are unhappy with that approach! Cheers, Carl Friedrich

3 3

Re: [pypy-dev] How to make a debug build of PyPy
by M A 10 Nov '21

10 Nov '21

Thank you so much. It worked. > On Nov 9, 2021, at 12:00 PM, pypy-dev-request(a)python.org wrote: > > Message: 2 > Date: Mon, 8 Nov 2021 22:13:44 +0100 > From: Carl Friedrich Bolz-Tereick <cfbolz(a)gmx.de> > To: pypy-dev(a)python.org > Subject: Re: [pypy-dev] How to make a debug build of PyPy > Message-ID: <bc073931-0e8d-8ed1-fd11-790f2b84cc50(a)gmx.de> > Content-Type: text/plain; charset=UTF-8; format=flowed > > this should work: > > rpython -Ojit --lldebug targetpypystandalone.py > > Cheers, > > CF > > On 08.11.21 20:41, M A wrote: >> Hi, I need to build PyPy with debug symbols in it. The documentation I found online says to use 'make lldebug' or 'make lldebug0'. This did not work. I am building the JIT version of PyPy. Any help or suggestions would be appreciated. Thank you.

1 0