Mailman 3 November 2019 - pypy-dev

PyPy Winter Sprint + cpyext
by Stefan Beyer Dec. 21, 2022

Dec. 21, 2022

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

upcoming release
by Matti Picus Nov. 26, 2019

Nov. 26, 2019

I would like to do a release of pypy soon. The biggest change from 7.2 is that we changed the SOABI to pypy27_pp73 (or pp36_pp73) to better hanlde packaging. Since a new release of pip will depend on our SOABI, I would like the release to happen soon. I am proposing this be "7.3.0" and not "8.0.0". Are there any outstanding issues we should fix before a new release? Packaging PyPy: I have been on changing our build/packaging on linux to produce a binary based on portable-pypy. My motivation is to enable projects like multibuild or cibuildwheels to download binary versions of PyPy from a single source. Is this a priority? The wininstaller branch provides an installer based on the CPython one. It works. Should we release it now? Is this a priority? Thanks for feedback Matti

3 2

Fwd: Feedback on Graduate School Research Idea Using Pypy
by yun bao Nov. 15, 2019

Nov. 15, 2019

Oops, replied directly. Forwarding to the mailing list. ---------- Forwarded message --------- From: yun bao <dayunbao(a)gmail.com> Date: Wed, Nov 13, 2019 at 9:57 AM Subject: Re: [pypy-dev] Feedback on Graduate School Research Idea Using Pypy To: Yury V. Zaytsev <yury(a)shurup.com> Thanks for all your feedback. You've given me a lot to think about. I'll probably send you some follow up questions regarding the topics you raised, if you don't mind. I was thinking that compiling PyPy to Wasm, and getting it running in the browser, would simply be the first step to doing real research, not as a research topic itself. I won't be starting grad school until next fall, so I have some time to play around with things and figure out exactly what I want to research. By the way, I was reading more about WebAssembly, and saw this: https://github.com/WebAssembly/binaryen. It might allow the removing of Emscripten from a toolchain that compiles an RPython interpreter to Wasm, since "It accepts input in WebAssembly-like form but also accepts a general control flow graph for compilers that prefer that." Maybe it's be possible to feed the control flow graphs the Annotator creates into Binaryen, and go straight to Wasm without ever first compiling to C? On Wed, Nov 13, 2019 at 12:00 AM Yury V. Zaytsev <yury(a)shurup.com> wrote: > An experimental WebAssembly backend is definitively an interesting idea, > but as you have realised, to get PyPy running in the browser, it's not > necessarily needed, as you can compile C to WebAssembly instead. > > Sure, a native backend sounds nicer, but from my PoV the real performance > questions lie elsewhere - so you've got an interpreter running under V8, > what next? Obviously, it's not performing very well. What are the > possibilities? > > Are you going to try to restructure the interpreter in a way, that your > language sematics become clearer to V8 so to say and it will be able to > more or less JIT your interpreter directly? Is it even possible, with > custom backend by avoiding Emscripten or otherwise? > > Or maybe it makes sense to first write a WebAssembly JIT backend keeping > the rest of the messy RPython -> C -> Emscripten -> WebAssembly toolchain > in place, and study how to JITs (PyPy generated JIT and V8 JIT) interact? > Is this interaction meaningful or harmful? Can it be turned into > productive cooperation? > > These are open questions, I don't know the answers, but what I want to say > is that just being able to port interpreters to the browser is already > possible via Emscripten, and in the context of PyPy there are other > (I think) much more interesting questions to study. > > ... and I wouldn't muddy the waters with Python 3 transition, working on > the questions above there is enough material for several PhDs, not even > speaking of a master thesis :-) > > On Tue, 12 Nov 2019, yun bao wrote: > > > I took a look at PyPyJS. I also spoke with some folks at Mozilla, and > have become very interested in WebAssembly. I think having > > WebAssembly as a target for RPython would be very useful. Based on what > I've learned recently, it should be possible to write an > > interpreter in RPython, translate it to C, then use Emscripten to > compile that to WebAssembly. However, if there was a WebAssembly backend > > for RPython, it would be possible to build interpreters than run > directly in the browser (potentially even PyPy.). That seems like an > > avenue worth investigating, doesn't it? I think while doing that, it > would be worth getting RPython compatible with Python 3. By the way, > > these are things I'm thinking about doing myself, not recommendations to > any of you. I'm still working on a solid thesis idea for my > > Master's. > > > > On Sat, Nov 9, 2019 at 12:25 AM Yury V. Zaytsev <yury(a)shurup.com> wrote: > > Maybe you should start by having a look at PyPyJS and then > exploring WebAssembly: > > https://github.com/pypyjs/pypyjs > > > > Sent from my iPad > > > > On 9. Nov 2019, at 03:40, yun bao <dayunbao(a)gmail.com> wrote: > > > > Hi, > > I'm planning to go to grad school for a Master's. I'm thinking about > researching something related to getting Python running > > in a web browser. All of the current implementations that allow you to > do that either transpile Python to JavaScript ahead of > > time, or do some kind of on-the-fly conversion from Python to > JavaScript. I'd like to write an actual Python engine that will > > run in the browser. As I've been mulling over this idea, I read about > Pypy. Given my limited knowledge of this domain > > (programming language engines that run in the browser) and the exact > nature of the problem, Pypy seems like the best route to > > achieving this. I'm still very much in the early stages of fleshing > this idea out, and besides the typical undergraduate > > Programming Language class, I haven't done much in the way of > programming language design or anything compiler related. So I > > have A LOT to learn. But I was hoping some of you who are Pypy experts > could give me a bit of advice on the feasibility of > > what I'm considering. > > > > Thank you, > > > > Andrew > > _______________________________________________ > > pypy-dev mailing list > > pypy-dev(a)python.org > > https://mail.python.org/mailman/listinfo/pypy-dev > > > > > > > > -- > Sincerely yours, > Yury V. Zaytsev

1 0

Feedback on Graduate School Research Idea Using Pypy
by yun bao Nov. 13, 2019

Nov. 13, 2019

Hi, I'm planning to go to grad school for a Master's. I'm thinking about researching something related to getting Python running in a web browser. All of the current implementations that allow you to do that either transpile Python to JavaScript ahead of time, or do some kind of on-the-fly conversion from Python to JavaScript. I'd like to write an actual Python engine that will run in the browser. As I've been mulling over this idea, I read about Pypy. Given my limited knowledge of this domain (programming language engines that run in the browser) and the exact nature of the problem, Pypy seems like the best route to achieving this. I'm still very much in the early stages of fleshing this idea out, and besides the typical undergraduate Programming Language class, I haven't done much in the way of programming language design or anything compiler related. So I have A LOT to learn. But I was hoping some of you who are Pypy experts could give me a bit of advice on the feasibility of what I'm considering. Thank you, Andrew

2 3

Taking billenstein-sierra buildbot offline
by Matt Billenstein Nov. 6, 2019

Nov. 6, 2019

It's a laptop with the infamous "battery swelling" issue - should have it back online next week. m -- Matt Billenstein matt(a)vazor.com http://www.vazor.com/

1 0