Mailman 3 June 2019 - pypy-dev

PyPy Winter Sprint + cpyext
by Stefan Beyer 21 Dec '22

21 Dec '22

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

better line number information in PyPy3
by Carl Friedrich Bolz 08 Aug '19

08 Aug '19

Hi all, This is a discussion mail about improving the traceback line number quality in PyPy3. Some context: CPython has started supporting negative line number offset from one bytecode to the next in its lnotab encoding, see here: https://bugs.python.org/issue26107 https://github.com/python/cpython/commit/f3914eb16d2#diff-cb296cc5109f5640f… (Please don't ask me why not a single test was added for this new feature.) However, the support is so far only theoretically possible, the Python 3.7/3.8 bytecode compiler does not seem to make use of the feature. Negative line number jumps occur quite regularly with continuation lines, eg like this: def foobar(a, b): return "" + (1, a, b, 2 ) If you call this function, the traceback will contain this line: File "x.py", line 5, in foobar 2 TypeError: can only concatenate str (not "tuple") to str Which is obviously misleading. A slightly more realistic example are function decorators. There, the execution order is regularly "backwards": first the function definition is executed, then the individual decorators are run. There is currently a hack to sometimes leave the line number on the line of the first decorator, leading to confusing tracebacks. Example: def works(f): return f def broken(f): assert 0 @works @broken @works def g(): pass Running this, you'll get the following traceback: Traceback (most recent call last): File "y.py", line 9, in <module> @works File "y.py", line 5, in broken assert 0 AssertionError Pointing you again to the wrong line. We could fix this problem rather easily by removing the code that prevents the bytecode compiler from emitting negative line number offsets. Should I implement this? Advantages: - A lot less confusing tracebacks in real-world cases Disadvantages: - Different behavior from CPython (but arguably more correct) - Tracing tools could potentially be confused - Potentially larger lnotab strings due to more jumping around Cheers, Carl Friedrich

2 1

Re: [pypy-dev] Is pypy 3.5 7.0 available on raspberry pi?
by Matt Billenstein 25 Jun '19

25 Jun '19

Hi all, You can rent aarch64 hardware on ec2 now - I built pypy3 7.1.1 on an a1.large (2 vCPUs, 4GB RAM, ~$37/mo on-demand) using an armhf chroot. It took about 140 minutes wall clock time - I did need to create a swap file as my first attempt failed with a memory error: real 139m41.253s user 118m12.756s sys 1m4.388s (armhf)ubuntu@ip-172-31-xx-xxx:~$ pypy3-build/opt/pypy3/bin/pypy3 Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, Jun 25 2019, 16:45:29) [PyPy 7.1.1-beta0 with GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``snow, snow, snow, snow'' >>>> 1 + 2 3 The same script on an i7 laptop with Ubuntu 18.04 took around 64 minutes: real 63m58.293s user 53m31.992s sys 0m17.487s I had also tried a build on a Raspberry Pi3b with an usb ssd as a swap disk (this machine only has 1GB of RAM) - this got pretty far, but failed after around 4 hours with a memory error. A Pi 4b with 4GB of RAM would compile it I imagine, so this might make a very cheap buildbot host for that purpose. m > On 2019-06-21 02:41, Matti Picus wrote: > > On 21/6/19 9:12 am, Gelin Yan wrote: > > Hi All > > > > I just noticed there are no links for downloading pypy 7.0 for Arm > > version. Does it imply I have to use the old version or the arm > > development has been suspended? > > > > Regards > > > > gelin yan > > > > While we still nominally support ARM32, we do not supply binaries. It > takes around 10 hours to translate on a qemu virtual machine with a x86 > host. The solution to cross-compile (scratchbox) is no longer supported. > We are exploring options for someone to host a aarch64 build machine > that can run a 32-bit chroot, which seems to be the best alternative. > Help sponsoring such work would be welcome. > > > Matti -- Matt Billenstein matt(a)vazor.com http://www.vazor.com/

1 1

How does PyPy’s performance compare with GraalPython’s performance?
by James Lu 24 Jun '19

24 Jun '19

PyPy is to RPython as GraalPython is to Graal. GraalPython is an implementation of Python on the Graal JIT. It employs what they call a “self-rewriting AST.” Graal is an AST interpreter. GraalPython uses the Truffle language framework and runs on the Truffle VM. What benchmarks should be run to compare GraalPython’s performance versus e.g. PyPy or CPython? From my observations, the Truffle VM seems to have zero overhead language interoperability— it can even inline across language boundaries!

1 0

Is pypy 3.5 7.0 available on raspberry pi?
by Gelin Yan 21 Jun '19

21 Jun '19

Hi All I just noticed there are no links for downloading pypy 7.0 for Arm version. Does it imply I have to use the old version or the arm development has been suspended? Regards gelin yan

2 1