Mailman 3 November 2015 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

Looking for clues to consistent Seg Fault in PyPy 2.6.1
by Jeff Doran 17 Feb '16

17 Feb '16

Hi, I've run out of options trying to find a Seg Fault which happens when running lxml under PyPy 2.6.1. This problem only occurs under PyPy as the rest of the code works fine under CPython 2.7. I've been in contact with the lxml dev team and they confirmed my problem, but could not determine where the cause of the Seg Fault lies. They suggested reaching out to you folks for ideas or a possible work around as the stack trace is always in libpypy-c.so and seems to be triggered by eventual garbage collection. Below is the gist of the email I sent to the lxml team earlier. It describes my ability to reproduce this Seg Fault using their supplied test suite. As of this writing, lxml is at version 3.5.0b1. I'd appreciate any feedback you might supply on how to deal with this error. Seg Faults are notoriously hard to find since they inevitably happen upstream for the crash. We are anxious to perform a complete evaluation of transitioning to PyPy as our Python platform of choice and this appears to be the last major blockage. Thanks for your help. - Jeff Doran ---------------- I'm running the lxml test suite under PyPy which I installed from here,https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-2.6.1-linux… . I've downloaded the current lxml src from https://github.com/lxml/lxml. I built lxml using 'python setup.py build --with-cython' *From ..lxml/src/lxml I run: * nosetests -v --pdb --nocapture Comparing with ElementTree 1.3.0 Comparing with cElementTree 1.3.0 TESTED VERSION: 3.5.0.beta1 Python: (major=2, minor=7, micro=10, releaselevel='final', serial=42) lxml.etree: (3, 5, 0, -99) libxml used: (2, 8, 0) libxml compiled: (2, 8, 0) libxslt used: (1, 1, 26) libxslt compiled: (1, 1, 26) lxml.html.tests.test_autolink.test_suite ... ok lxml.html.tests.test_basic.test_suite ... ok test_allow_tags (lxml.html.tests.test_clean.CleanerTest) ... ok test_clean_invalid_root_tag (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_excluded (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_included (lxml.html.tests.test_clean.CleanerTest) ... ok lxml.html.tests.test_clean.test_suite ... ok lxml.html.tests.test_diff.test_suite ... ok test_body (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... ok test_broken_attribute (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... fish: Job 1, “nosetests -v --pdb --nocapture” terminated by signal SIGSEGV (Address boundary error) ``` Running these tests under gdb reveals the following partial stack trace: #0 0x00007ffff57fa08a in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #1 0x00007ffff57fa110 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #2 0x00007ffff5776be3 in PyPyWeakref_LockObject () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #3 0x00007ffff0337399 in __pyx_f_4lxml_5etree__isProxyAliveInPypy.isra.265 () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #4 0x00007ffff033fbfb in __pyx_f_4lxml_5etree_attemptDeallocation () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #5 0x00007ffff033fd70 in __pyx_tp_dealloc_4lxml_5etree__Element () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #6 0x00007ffff62186f9 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #7 0x00007ffff57aec90 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #8 0x00007ffff57fa670 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #9 0x00007ffff5fa0da8 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #10 0x00007ffff5f89a31 in ?? ()

6 17

Leysin Winter sprint?
by Armin Rigo 01 Jan '16

01 Jan '16

Hi all, Due to public pressure :-) I'm trying to organize this winter's sprint, in Leysin, Switzerland. This is a fully public sprint of 7 days, with possible skiing. If you'd like to come, please reply to this e-mail with dates that are ok for you. I will give some priority to core people, but anyone else is welcome to have preferences too. Traditionally, it is around mid-January, but this year as far as I understand there is some push to have it in February or March instead---which is fine too. The following week should be avoided if possible, as it is holidays for the canton's schools: 20-28 Feb 2016. A bientôt, Armin.

7 11

Multiprocessing - CPython and PyPy
by Luis José Novoa 25 Nov '15

25 Nov '15

Hi everyone. Im trying to solve some problems in parallel using the multiprocessing module in Python 2.7. A general model is built using CPython and then the subproblems are solved in parallel returning results in a queue. This is currently working fine. I would like to solve the subproblems using PyPy to increase speed. I found http://project-trains.tumblr.com/post/102076598295/multiprocessing-pypy , but there it says that the procedure only works with CPython 3.4. I wonder is there is any clean direct way to do this. Appreciate any help. -- Luis J. Novoa

4 9

debugging extension segfault
by Robin Becker 24 Nov '15

24 Nov '15

I hope this is the right place to ask, but if not just tell me to go away. At the request of a user I'm trying to get reportlab up to scratch for pypy. I have some fairly insignificant bugs in the tests related to Cpython-pypy differences and one C extension that crashes. Firstly what is the best way to approach debugging an extension segfault. Do I need to build pypy with debugging turned on or can I just build the extension like that? Are there any guidelines for making c extensions pypy friendly? Secondly how to cope with some fairly simple testing problems. One of my extensions appears to work, but clearly attempting to verify reference counts is failing ie from sys import getrefcount, _getframe doesn't work. I suppose there's no real point in trying to figure out refcounts for pypy. I have some errors related to co_filename eg > if os.path.splitext(obj.__code__.co_filename)[0]==modBn: > AttributeError: 'builtin-code' object has no attribute 'co_filename' but it's not clear to me if it's even possible to get the filename. Lastly what is the right way to get the platform as pypy not cpython? -- Robin Becker

4 4

ARM failing test
by Antonio Cuni 23 Nov '15

23 Nov '15

Hi David, I am developing the "faster-rstruct" branch, which aims to speedup struct.unpack by reading the values at once from memory, instead of byte-by-byte as it's doing right now. The branch works fine on x86 but fails on armhf (see e.g. http://buildbot.pypy.org/summary/longrepr?testname=AppTestStruct.%28%29.tes…). I suspect a big/little endian issue. Two questions: 1) I know that ARM CPUs can be either little or big endiam. What is the case for our armhf machine? 2) is it possible to have ssh access to one or more of our ARM machines so I can test easily? thank you :) ciao, Anto

3 3

How to convert C++ char*(contain \0) to python string in PyPy?
by Yicong Huang 18 Nov '15

18 Nov '15

We used ffi.string() to convert C++ char* to pypy string. But recently, we found out ffi.string() used '\0' as the indicator of string termination. And it bring some problems of converting a binary string. Our case is we read char*(buf) from a binary file, and buf might contain '\0'. Using ffi.string() we were not able to convert completed buf to PyPy string. We've tried ffi.string(buf, length), but it also did not work. Are there any solutions?

2 2

PyPy 4.0.1 tarballs avaiable
by Matti Picus 18 Nov '15

18 Nov '15

Please try them out, I want to release in the next few days. https://bitbucket.org/pypy/pypy/downloads Also any comments (or edits) to the release notice would be nice http://pypy.readthedocs.org/en/latest/release-4.0.1.html Matti P.S. here are the hashes (without PPC), I will modify pypy.org/downloads closer to the release itself f6721e62f4ba1cdc4cc5ad719369e359 pypy-4.0.1-linux64.tar.bz2 fe8106ac3919c7b4be2766944326a624 pypy-4.0.1-linux-armel.tar.bz2 823b8a457f4c48ebdb8e1ee607b0a893 pypy-4.0.1-linux-armhf-raring.tar.bz2 e45728d413aa88963d4462ebcfaff6ea pypy-4.0.1-linux-armhf-raspbian.tar.bz2 d1d03aa44df354a3f589473a51406795 pypy-4.0.1-linux.tar.bz2 67ac82d88aaaef8c3074e68d700f3968 pypy-4.0.1-osx64.tar.bz2 f5b35ebedee2fa4fdfee82733be59996 pypy-4.0.1-src.tar.bz2 6b81d79886b215a877fee7624883ee94 pypy-4.0.1-src.zip dc6573828ee5c82df18f9035f3b19edb pypy-4.0.1-win32.zip f723ed8aa51c98dcad703ff87c32ade3a54f0101 pypy-4.0.1-linux64.tar.bz2 947a24eff8f50705b403bd67f3357e135f071036 pypy-4.0.1-linux-armel.tar.bz2 bbcc04d524a5e7634173a7272b233f675a719bbf pypy-4.0.1-linux-armhf-raring.tar.bz2 adf73fb7bb8001f549472eba5fee970c1800bc56 pypy-4.0.1-linux-armhf-raspbian.tar.bz2 b306273b86a86ef3c624ecea7e48e45f660e1ef5 pypy-4.0.1-linux.tar.bz2 75286d96b22040723fe1de061d3df43cea712903 pypy-4.0.1-osx64.tar.bz2 805e9a81528a905afa571aa33cc66c650ae4e9f4 pypy-4.0.1-src.tar.bz2 5f29a916eba5f2b31affa56753b7ca108718d9be pypy-4.0.1-src.zip 992b110210e556cd6803c2ca3837bec82a09aa15 pypy-4.0.1-win32.zip

1 0

Performance regression from 2.6.1 to 4.0.0 ?
by Tobias Oberstein 17 Nov '15

17 Nov '15

Hi, I've done some measurements comparing JSON and MsgPack serialization/deserialization performance https://github.com/oberstet/scratchbox/tree/master/python/serperf MsgPack deserialization seems significantly slower (>30%). Note: the MsgPack library used runs pure Python code when on PyPy. Is this expected/known? Cheers, /Tobias

3 3

PyPy 4.0.1 bug-fix release
by Matti Picus 15 Nov '15

15 Nov '15

After fixing the critical bugs 2180 (unrolling double loops) and 2183 (ssl memory leak), should we release a bug-fix PyPy 4.0.1? There were also some nice performance enhancements since 4.0.0, esp. list and array slicing Any thoughts if there are critical outstanding issues blocking a bug-fix PyPy 4.0.1 very soon? Matti

5 4