Mailman 3 January 2022 - pypy-dev

PyPy Winter Sprint + cpyext
by Stefan Beyer 20 Dec '22

20 Dec '22

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

PyPy v7.3.8 rc1 now available
by Matti Picus 11 Feb '22

11 Feb '22

This is a release of 4 versions of python: 2.7, 3.7, 3.8 (no longer beta) and 3.9 (beta quality). Release note: https://doc.pypy.org/en/latest/release-v7.3.8.html Downloads: https:// checksums: https://www.pypy.org/checksums.html As soon as the CDN network updates the https://downloads.python.org/pypy/versions.json file, you can use them in github actions CI. Please try these out and report any problems. Note that we are having issues with the ARM8-64bit buildbot, so we are not releasing binary packages for that platform. Hopefully we will have an alternative for the final release. Matti

2 3

How to declare a variable as a void** type
by M A 30 Jan '22

30 Jan '22

Hi, I'm working on an libffi function called ffi_closure_alloc() that takes for its second argument a variable of type void**. Is there a way to declare a Rpython variable as a point to a pointer to a void?

1 0

Installation layout of the PyPy 3.8 Fedora package
by Miro Hrončok 27 Jan '22

27 Jan '22

Hello PyPy folks, I co-maintain the PyPy and CPython interpreter packages in Fedora and I am excited but a bit puzzled by a certain change that happened in PyPy 3.8. The installation layout of PyPy 3.8 was changed to be more like the CPython layout: https://doc.pypy.org/en/latest/release-v7.3.6.html > The 3.8 package now uses the same layout as CPython, and many of the > PyPy-specific changes to sysconfig, distutils.sysconfig, > and distutils.commands.install.py have been removed. The stdlib now is > located in <base>/lib/pypy3.8 on posix systems, and in <base>/Lib on Windows. > The include files on windows remain the same, on posix they are in > <base>/include/pypy3.8 . Note we still use the pypy prefix to prevent mixing > the files with CPython (which uses python). Up until now (e.g. with PyPy 3.7) the installation layout of the Fedora package looked like this: /usr/lib64/pypy3.7/ ├── bin │ └── ... ├── include │ └── *.h ├── lib_pypy │ └── ... ├── lib-python │ └── 3 │ └── ... └── site-packages └── ... /usr/bin/pypy3.7 -> /usr/lib64/pypy3.7/bin/pypy3.7 (sometimes) /usr/bin/pypy3 -> /usr/bin/pypy3.7 We basically took the advice from https://doc.pypy.org/en/latest/build.html > To install PyPy system wide on unix-like systems, it is recommended to put > the whole hierarchy alone (e.g. in /opt/pypy) and put a symlink to the pypy > executable into /usr/bin or /usr/local/bin. Where we used "/usr/lib64/pypy3.7" instead of "/opt/pypy". Several different versions of pypy (e.g. 3.6 and 3.7) could be installed alongside each other, as long as only one of them "owned" /usr/bin/pypy3. Now with PyPy 3.8, if we are to keep this scheme, the entire tree gets a bit more nested: /usr/lib64/pypy3.8/ ├── bin │ └── ... ├── include │ ├── PyPy.h │ └── pypy3.8 │ └── *.h └── lib └── pypy3.8 ├── ... └── site-packages └── ... (/usr/bin setup remains the same) This e.g. means that the standard library is installed in: /usr/lib64/pypy3.8/lib/pypy3.8/ Or the header files are in: /usr/lib64/pypy3.8/include/pypy3.8/ Double namespacing like this seems a bit overengineered, and since the idea was to resemble CPython, I was aiming at this setup: /usr/ ├── bin │ └── ... ├── include │ └── pypy3.8 │ └── *.h └── lib(64) └── pypy3.8 ├── ... └── site-packages └── ... Which would be the exact copy of our CPython setup | sed s/python/pypy/ However, I found 4 problems: 1) The PyPy install scheme only has "lib", not "lib64" I think we could sort this by a tiny patch that we used to carry in CPython 3.8 before configurable platlibdir was introduced in 3.9. 2) The /usr/include/PyPy.h file is *not* namespaced with PyPy Python version I don't know if we even need to ship this file. It seems deprecated. Maybe it can be in /usr/include/pypy3.8/PyPy.h? 3) /usr/bin/pypy3.8 is a symbolic link to /usr/bin/pypy3 I would expect it to be the other way around, so both /usr/bin/pypy3.X and /usr/bin/pypy3.Y can co-exist and /usr/bin/pypy3 is an (optional) symbolic link to one of them (as is the situtaion with CPython). I guess I can somehow reverse the order manually, but I have not tested if it breaks anything. 4) The /usr/bin/libpypy3-c.so file is *not* namespaced and seems misplaced This is the one thing I cannot wrap my head around. I do not wish to install this file "globally" to /usr/bin for two reasons: a) it conflicts between different PyPy Python versions b) it ends up on $PATH but segfaults when executed So, I am asking on this list: How should we package PyPy 3.8 for Fedora? Do we need to install it to /usr/lib64/pypy3.8/ prefix even thou most (but not all) of the directories within would contain a "pypy3.8" subdirectory? Or is there a better option I do not yet see? Thanks for your help. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

5 9

MoreVMs 2022 : Workshop on Modern Language Runtimes, Ecosystems, and VMs
by Rodrigo Bruno 17 Jan '22

17 Jan '22

====================================================================== Call for Extended Abstracts and Talks: MoreVMs’22 6th Workshop on Modern Language Runtimes, Ecosystems, and VMs Co-located with ‹Programming›’22 March 22nd, 2022, Porto, Portugal https://2022.programming-conference.org/home/MoreVMs-2022#Call-for-Talks-an… ====================================================================== Following six previous successful editions, the MoreVMs’22 workshop aims to bring together industrial and academic programmers to discuss the design, implementation, and usage of modern languages and runtimes. This includes aspects such as reuse of language runtimes, modular implementation, language design, and compilation strategies. By bringing together both researchers and practitioners, the workshop aims to enable a diverse discussion on how languages and runtimes are currently being utilized, and where they need to improve further. In addition to conventional workshop-style submissions, MoreVMs also accepts (and encourages) submissions that present early-stage work and emerging ideas. Relevant topics include, but are definitely not limited to, the following: - Extensible VM design (compiler- or interpreter-based VMs) - Reusable components (e.g. interpreters, garbage collectors, ...) - Static and dynamic compilation techniques - Techniques for targeting high-level languages such as JavaScript - Interoperability between languages - Tooling support (e.g. debugging, profiling, etc.) - Programming language development environments - Case studies of existing language implementation approaches - Language implementation challenges and trade-offs - Surveys and usage reports to understand usage in the wild - Ideas for more predictable performance - Ideas for how VMs could take advantage of new hardware features - Ideas for how we should build languages in the future Workshop Format and Submissions ------------------------------- We welcome presentation proposals in the form of extended abstracts (2 to 4 pages long), talk proposals (title and 400 words abstract), and blog posts, discussing new techniques, insights, experiences, works-in-progress, as well as future visions, from either an academic or industrial perspective. The extended abstracts and talk proposals, and if the speakers wish, their slides, will be published on the workshop's website. Alternatively, extended abstracts can be published as part of the companion of ‹Programming›’22 in the ACM DL. Publication in the ACM DL is conditional on the acceptance by the program committee. Please note that MoreVMs’22 is organized as an academic workshop, and as such, speakers will be required to register for the workshop. We regret that we are unable to cover registration, travel, or accommodation costs for authors. Important Dates --------------- Extended abstract and talk submissions: 2022-01-27 Author notification: 2022-02-15 Camera Ready: 2022-03-01 Workshop: 2022-03-22 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC-12:00 hour. Organizers ---------- Rodrigo Bruno, INESC-ID / Técnico, ULisboa, Portugal Michael Engel, NTNU, Norway -- rodrigo-bruno.github.io

1 0

Running PyPy on top of CPython
by M A 11 Jan '22

11 Jan '22

Hello, I read here (https://rpython.readthedocs.io/en/latest/rpython.html) that PyPy is runnable on top of CPython. How do I start PyPy so it runs in CPython?

5 9

Re: [pypy-dev] Running PyPy on top of CPython
by M A 10 Jan '22

10 Jan '22

> On 9/1/22 9:50 pm, M A wrote: >> Hello, I read here (https://rpython.readthedocs.io/en/latest/rpython.html) that PyPy is runnable on top of CPython. How do I start PyPy so it runs in CPython? > > > python2 pypy/bin/pyinteractive.py > > > You can get help with > > python2 pypy/bin/pyinteractive.py --help > > > It takes ~20 secs to startup. To drop into rpython, hit ctrl-c, to > return to interpreted PyPy on top of CPython hit ctrl-d > > > In rpython you have access to "space". > > > > Matti Thank you very much for this information. It really helped me.

1 0

Calling a C function from PyPy
by M A 10 Jan '22

10 Jan '22

Hi, I'm trying to call my own C function from a small RPython program. When I try to build the program I see this error: [translation:ERROR] Exception: stand-alone program entry point must return an int (and not, e.g., None or always raise an exception). This is the entire RPython program: from rpython.rtyper.lltypesystem import rffi, lltype from rpython.translator.tool.cbuild import ExternalCompilationInfo info = ExternalCompilationInfo(includes=["./myheader.h"], libraries=["./my_functions.so"]) print_thread_mode = rffi.llexternal("print_thread_mode", [lltype.Void], lltype.Void, compilation_info=info) def entry_point(argv): print_thread_mode() return 0 def target(*args): return entry_point The entry_point function is returning an int (0 here). If I remove the call to print_thread_mode() the program compiles. The C function is declared in a .c file. I compiled that file to .so and was able to use it using ctypes in CPython. This is its prototype: void print_thread_mode(void). It simply prints some text. Is there something I am missing?

1 0

pypy-dev January 2022