Mailman 3 August 2022 - pypy-dev

Re: Injecting an exception into another thread
by Audrey Dutcher 09 May '23

09 May '23

Hi! Sorry for the delay, I didn't realize I needed to be subscribed to the mailing list to participate in conversations. My use case is that I have a gui application which runs arbitrary tasks as background threads, and needs to provide a cancel button to interrupt them. The tasks are often legacy code and/or are very difficult to find reasonable points to insert checks for interruption. Another use case would be implementing a python shell in a gui. If you want to maintain responsiveness while commands execute, they must be able to run in another thread (or another process, but this is not ideal if the shell is supposed to interact with the internals of your application). Interrupting lines with ctrl-c requires being able to send an exception to worker threads. Hope this helps! Thanks, - Audrey

3 2

PyPy Winter Sprint + cpyext
by Stefan Beyer 20 Dec '22

20 Dec '22

Hey everyone! Those of you who attended or followed one of the last two winter sprints in Leysin might know me, the others probably won't. I'm (still) a Master's student writing my thesis on PyPy's current issue with cross-heap cycles when using cpyext. The main point is, that they are not collected and stay around as floating garbage, even if they are not reachable any more. Correct me if I'm wrong, but I didn't notice anyone else working on that topic (and/or committing something) since I've picked it up two years ago (yeah, I know, that long ago...). I saw that you would be working on "cpyext performance and completeness" during the current winter sprint this week and thought that this might also concern my thesis. So I thought I'll give you an update. I recently pushed my current (non-optimized, breaking-some-tests, but more or less working) implementation of the "CPython-style" GC-extension for cpyext. It is still in an early stage and not all cases (legacy and non-legacy-finalizers, weakrefs) are handled. However, I picked up some pace during the last couple of weeks and I'm determined to finish this implementation during the following weeks (some quirks with the tests have been haunting me, but I think I figured most of them out by now). After that, I will implement a second alternative implementation (to compare to, mostly for the sake of my thesis), which will take some more time. I also added a couple of fancy test cases (the so called "dot-tests"), so that we can test complex object graphs a little bit easier (and also because it was kind of cool to have another language inside PyPy/RPython, even if it was only for the tests...), with more test cases to come (the current ones are a bit messy). None of the new changes concerning low-latency applications are currently integrated, but that should not be too hard. I guess it won't make much sense for me to join you at this year's winter sprint, especially as I won't be able to get there before Thursday, but I might be able to join you over the IRC channel (or some other form of communication if you like). If there is anything that is worth discussing please let me know! Also feel free to comment on my code, but beware that I might change some things once I try to do some optimizations (probably not many, but at least fix the worst issues) and make it a little bit more readable. You can find my code on the cpyext-gc-cycle branch.<https://bitbucket.org/pypy/pypy/commits/branch/cpyext-gc-cycle> Looking forward to hearing from you! Greetings, Stefan

2 2

Performance tuning, GC logs
by AJ R 29 Aug '22

29 Aug '22

Hi, What's the recommended way to improve my program's performance, in particular wrt garbage collection? I see jitviewer and vmprof, but I'm not sure if either is still maintained. I've noticed speed improvements after playing with the gc nursery size, but that was trial-and-error. Thanks. Albert-Jan

2 1

Please help to solve the problem with Cyrillic UTF-8
by Максим Кужильный 25 Aug '22

25 Aug '22

Hello! Please help to solve the problem with Cyrillic UTF-8! Windows 10 Python 3.9.10 (b332b321bbaa72bffb0207da5b7fe4c38047d3b2, Mar 16 2022, 16:03:21) [PyPy 7.3.9 with MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>>> print ("АБВвба") UnicodeEncodeError: 'utf-8' codec can't encode character '\udc80' in position 8: surrogates not allowed Visual Studio Code + PyPy 7.3.9 print ("АБВвба") ╨Р╨С╨Т╨▓╨▒╨░ != Normal output Python 3.10.5 (tags/v3.10.5:f377153, Jun 6 2022, 16:14:13) [MSC v.1929 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> print ("АБВвба") АБВвба == Normal output Because of this behavior, all actions on strings containing Cyrillic are incorrect. Is it possible to solve this problem? I tried using "setlocale" by analogy with C, assuming that the code is translated to C, but this does not work in PyPy. from locale import setlocale , LC_ALL setlocale ( LC_ALL , "ru_RU.UTF-8" ) Maybe it is necessary to add a localization check in the PyPy sources. #include <stdio.h> #include <conio.h> #include <locale.h> int main () { setlocale ( LC_ALL , "ru_RU.UTF-8" ); printf ( "АБВвба" ); _getch (); } I can't do it myself since I just started learning C, I don't have enough knowledge for this, but I really want to learn) Interestingly, when compiling to GCC (MinGW or MinGW64), Cyrillic support does not work, and Cyrillic output to C is not correct, but if you compile Clang (LLVM), everything works correctly. I am grateful in advance for any answer! Thanks! With great respect! Max.

1 0

32bit Windows build
by enquiries＠oneledger.co.uk 11 Aug '22

11 Aug '22

Hi, Any chance of making a 32bit Windows build of PyPy2.7-v7.3.9 available for download? I have a project that connects to a legacy database using a 32bit ODBC driver so sadly switching to 64bit isn't an option for me. I'm currently running with PyPy2.7-v7.3.3 so it's not a disaster if this is too much hassle but it would be nice to be up to date with enhancements. Best regards, Steve

2 3

Jitviewer and performance tuning
by AJ R 08 Aug '22

08 Aug '22

Hi, I converted a long running program (~52k LOC) from python 2.7 to python 3.8. I run my tests on Window 10 (64) against pypy 3.8 (7.3.7) and cpython 3.8. Now I'm profiling the code with cProfile + gprof2dot. I noticed that increasing PYPY_GC_NURSERY improved the program's performance, but it's still only marginally faster than under Cpython. I set PYPYLOG to generate a Jitviewer-compatible file (~ 17Mb), but it seems that this tool is no longer maintained. Is that correct? What can I do to get more insight into what the JIT is doing? And what GC vars should generally be used? Thanks in advance for your help. Best wishes, Albert-Jan

1 0

OpenSSL 3: Inconsistent CPython/PyPy behavior wrt. unsupported hash types: incorrect results
by Miro Hrončok 04 Aug '22

04 Aug '22

Hello, we have recently switched PyPy to OpenSSL 3 in Fedora and we see an inconsistent behavior wrt. unsupported hash types, such as md4. This is CPython (OpenSSL 3): ====================================================================== Python 3.9.13 (main, Jul 22 2022, 00:00:00) [GCC 12.1.1 20220628 (Red Hat 12.1.1-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import hashlib >>> m = hashlib.new('md4') Traceback (most recent call last): File "/usr/lib64/python3.9/hashlib.py", line 160, in __hash_new return _hashlib.new(name, data, **kwargs) ValueError: [digital envelope routines] unsupported During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib64/python3.9/hashlib.py", line 166, in __hash_new return __get_builtin_constructor(name)(data) File "/usr/lib64/python3.9/hashlib.py", line 123, in __get_builtin_constructor raise ValueError('unsupported hash type ' + name) ValueError: unsupported hash type md4 ====================================================================== An exception is raised -- md4 is not supported. And this is PyPy (OpenSSL 3): ====================================================================== Python 2.7.18 (f1d28f87e6d8, Jul 22 2022, 16:05:41) [PyPy 7.3.9 with GCC 12.1.1 20220628 (Red Hat 12.1.1-3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>> import hashlib >>>> m = hashlib.new('md4') >>>> m.digest() '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' ====================================================================== Python 3.9.12 (dd85a1aa7d80, Jul 22 2022, 16:05:26) [PyPy 7.3.9 with GCC 12.1.1 20220628 (Red Hat 12.1.1-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>>> import hashlib >>>> m = hashlib.new('md4') >>>> m.digest() b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' ====================================================================== No exception here, returns all zeros, which is incorrect result, see bellow: This is CPython and PyPy both with OpenSSL 1.1: ====================================================================== >>>> import hashlib >>>> m=hashlib.new('md4') >>>> m.digest() b'1\xd6\xcf\xe0\xd1j\xe91\xb7<Y\xd7\xe0\xc0\x89\xc0' ====================================================================== As you can see, PyPy with OpenSSL 3.0 not only does not raise an exception, it returns incorrect results. Is this a bug? Or is it a known limitation? Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0