Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan
On 4/2/19 10:36, Stefan Beyer wrote:
Hi Stefan. It has been a while, I hope all is good by you. Recently there was some interest in looking at the work you did in the cpyext-gc-cycle branch toward getting it merged into PyPy. Did you ever get any further than the published code? I merged the current py3.8 branch into cpyext-gc-cycle and immediately ran into problems (see below). It would be great to keep pushing this forward. Would you like to pick this up again or review changes as needed? Any thoughts? Matti N.B: PyUnicodeObject has tp_itemsize==0 when in reality it sometimes allocates extra room for data [0]. My fix in [1] didn't help. [0] https://foss.heptapod.net/pypy/pypy/-/issues/3772 [1] https://foss.heptapod.net/pypy/pypy/-/commit/2d53e90b42a803bfac8ca1fe7b8f704...
Hi Matti, it has definitely been a while! Sorry for the late response, I was a bit busy at work. Yes I would love to see my code finally put into action! At the moment I could only contribute by reviewing changes or consulting the development, as my main topic of research at my current employer (AIT) is telehealth / digital health, not language development, and we are quite busy with that right now. But as we are using Django and, to a limited extend, also some extension modules, like NumPy, for it, there might be a chance for a cooperation. Either directly, or with one of our customers / spin offs. I can keep you updated on that, just don‘t expect anything to happen too soon. Just give me a few days to set up my dev machine again and I will get back to you with regards to your merge and your changes to the code. I don’t know if I can make it before Christmas, but maybe during the holidays. Greetings, Stefan
On 4/2/19 10:36, Stefan Beyer wrote:
Hi Stefan. It has been a while, I hope all is good by you. Recently there was some interest in looking at the work you did in the cpyext-gc-cycle branch toward getting it merged into PyPy. Did you ever get any further than the published code? I merged the current py3.8 branch into cpyext-gc-cycle and immediately ran into problems (see below). It would be great to keep pushing this forward. Would you like to pick this up again or review changes as needed? Any thoughts? Matti N.B: PyUnicodeObject has tp_itemsize==0 when in reality it sometimes allocates extra room for data [0]. My fix in [1] didn't help. [0] https://foss.heptapod.net/pypy/pypy/-/issues/3772 [1] https://foss.heptapod.net/pypy/pypy/-/commit/2d53e90b42a803bfac8ca1fe7b8f704...
Hi Matti, it has definitely been a while! Sorry for the late response, I was a bit busy at work. Yes I would love to see my code finally put into action! At the moment I could only contribute by reviewing changes or consulting the development, as my main topic of research at my current employer (AIT) is telehealth / digital health, not language development, and we are quite busy with that right now. But as we are using Django and, to a limited extend, also some extension modules, like NumPy, for it, there might be a chance for a cooperation. Either directly, or with one of our customers / spin offs. I can keep you updated on that, just don‘t expect anything to happen too soon. Just give me a few days to set up my dev machine again and I will get back to you with regards to your merge and your changes to the code. I don’t know if I can make it before Christmas, but maybe during the holidays. Greetings, Stefan
participants (2)
-
Matti Picus -
Stefan Beyer