Mailman 3 December 2021 - pypy-dev

PyPy Winter Sprint + cpyext
by Stefan Beyer Dec. 21, 2022

Dec. 21, 2022

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

Installation layout of the PyPy 3.8 Fedora package
by Miro Hrončok Jan. 27, 2022

Jan. 27, 2022

Hello PyPy folks, I co-maintain the PyPy and CPython interpreter packages in Fedora and I am excited but a bit puzzled by a certain change that happened in PyPy 3.8. The installation layout of PyPy 3.8 was changed to be more like the CPython layout: https://doc.pypy.org/en/latest/release-v7.3.6.html > The 3.8 package now uses the same layout as CPython, and many of the > PyPy-specific changes to sysconfig, distutils.sysconfig, > and distutils.commands.install.py have been removed. The stdlib now is > located in <base>/lib/pypy3.8 on posix systems, and in <base>/Lib on Windows. > The include files on windows remain the same, on posix they are in > <base>/include/pypy3.8 . Note we still use the pypy prefix to prevent mixing > the files with CPython (which uses python). Up until now (e.g. with PyPy 3.7) the installation layout of the Fedora package looked like this: /usr/lib64/pypy3.7/ ├── bin │ └── ... ├── include │ └── *.h ├── lib_pypy │ └── ... ├── lib-python │ └── 3 │ └── ... └── site-packages └── ... /usr/bin/pypy3.7 -> /usr/lib64/pypy3.7/bin/pypy3.7 (sometimes) /usr/bin/pypy3 -> /usr/bin/pypy3.7 We basically took the advice from https://doc.pypy.org/en/latest/build.html > To install PyPy system wide on unix-like systems, it is recommended to put > the whole hierarchy alone (e.g. in /opt/pypy) and put a symlink to the pypy > executable into /usr/bin or /usr/local/bin. Where we used "/usr/lib64/pypy3.7" instead of "/opt/pypy". Several different versions of pypy (e.g. 3.6 and 3.7) could be installed alongside each other, as long as only one of them "owned" /usr/bin/pypy3. Now with PyPy 3.8, if we are to keep this scheme, the entire tree gets a bit more nested: /usr/lib64/pypy3.8/ ├── bin │ └── ... ├── include │ ├── PyPy.h │ └── pypy3.8 │ └── *.h └── lib └── pypy3.8 ├── ... └── site-packages └── ... (/usr/bin setup remains the same) This e.g. means that the standard library is installed in: /usr/lib64/pypy3.8/lib/pypy3.8/ Or the header files are in: /usr/lib64/pypy3.8/include/pypy3.8/ Double namespacing like this seems a bit overengineered, and since the idea was to resemble CPython, I was aiming at this setup: /usr/ ├── bin │ └── ... ├── include │ └── pypy3.8 │ └── *.h └── lib(64) └── pypy3.8 ├── ... └── site-packages └── ... Which would be the exact copy of our CPython setup | sed s/python/pypy/ However, I found 4 problems: 1) The PyPy install scheme only has "lib", not "lib64" I think we could sort this by a tiny patch that we used to carry in CPython 3.8 before configurable platlibdir was introduced in 3.9. 2) The /usr/include/PyPy.h file is *not* namespaced with PyPy Python version I don't know if we even need to ship this file. It seems deprecated. Maybe it can be in /usr/include/pypy3.8/PyPy.h? 3) /usr/bin/pypy3.8 is a symbolic link to /usr/bin/pypy3 I would expect it to be the other way around, so both /usr/bin/pypy3.X and /usr/bin/pypy3.Y can co-exist and /usr/bin/pypy3 is an (optional) symbolic link to one of them (as is the situtaion with CPython). I guess I can somehow reverse the order manually, but I have not tested if it breaks anything. 4) The /usr/bin/libpypy3-c.so file is *not* namespaced and seems misplaced This is the one thing I cannot wrap my head around. I do not wish to install this file "globally" to /usr/bin for two reasons: a) it conflicts between different PyPy Python versions b) it ends up on $PATH but segfaults when executed So, I am asking on this list: How should we package PyPy 3.8 for Fedora? Do we need to install it to /usr/lib64/pypy3.8/ prefix even thou most (but not all) of the directories within would contain a "pypy3.8" subdirectory? Or is there a better option I do not yet see? Thanks for your help. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

5 9

Feature request: support for class variables and class functions
by M A Dec. 18, 2021

Dec. 18, 2021

Hi I ask that support for class variables and class functions be added to RPython. I was recently trying to work on some code for PyPy and found out this feature wasn't supported. This feature is a nice feature to have and would benefit the PyPy community. Thank you.

1 0