Mailman 3 February 2016 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

mingwpy
by anatoly techtonik 24 Feb '16

24 Feb '16

I am not sure if CFFI list a better place to post about this one, but it seems to me that you may be interested in the initiative to create GCC that can link to MS VS runtimes. Probably it can be used to cross-compile right stubs for interoperability with C code on Windows platform. https://mingwpy.github.io/ https://groups.google.com/forum/#!topic/mingwpy/z6TMklH56w8 -- anatoly t.

1 0

Google Summer of Code ideas
by Kunal Grover 23 Feb '16

23 Feb '16

Hi, I am interested to work with PyPy on a GSoC project. PyPy is still not present as a registered subproject on the Python page, so I wanted to ask on the list if someone is willing to mentor me. Are the tasks listed on the project ideas page up to date? What would be some of the projects that would be very beneficial for PyPy? Thank you. Kunal Grover

2 3

Partial Escape Analysis and Scalar Replacement
by George Papanikolaou 22 Feb '16

22 Feb '16

Hey list, I'd like to announce that I will attempt to implement an optimization technique in PyPy as part of my thesis for my CS diplomma. In addition to the local advisor from my uni, Carl Friedrich Bolz (@cfbolz) accepted to "mentor" me for the attempt. I'll be reporting the progress frequently here (so you will hear soon more details) and of course I'll also be availabe on the IRC channel, which I'll also use for the daily communication with Carl and for any other questions or problems I may have with the codebase. The optimization is based on the paper "Partial Escape Analysis and Scalar Replacement on Java" by L. Stadler, T. Würthinger and H. Mössenböck. Feel free to add any comments, remarks, ideas, etc. Have a good day. Cheers, George -- George 'papanikge' Papanikolaou

4 3

Looking for clues to consistent Seg Fault in PyPy 2.6.1
by Jeff Doran 17 Feb '16

17 Feb '16

Hi, I've run out of options trying to find a Seg Fault which happens when running lxml under PyPy 2.6.1. This problem only occurs under PyPy as the rest of the code works fine under CPython 2.7. I've been in contact with the lxml dev team and they confirmed my problem, but could not determine where the cause of the Seg Fault lies. They suggested reaching out to you folks for ideas or a possible work around as the stack trace is always in libpypy-c.so and seems to be triggered by eventual garbage collection. Below is the gist of the email I sent to the lxml team earlier. It describes my ability to reproduce this Seg Fault using their supplied test suite. As of this writing, lxml is at version 3.5.0b1. I'd appreciate any feedback you might supply on how to deal with this error. Seg Faults are notoriously hard to find since they inevitably happen upstream for the crash. We are anxious to perform a complete evaluation of transitioning to PyPy as our Python platform of choice and this appears to be the last major blockage. Thanks for your help. - Jeff Doran ---------------- I'm running the lxml test suite under PyPy which I installed from here,https://bitbucket.org/squeaky/portable-pypy/downloads/pypy-2.6.1-linux… . I've downloaded the current lxml src from https://github.com/lxml/lxml. I built lxml using 'python setup.py build --with-cython' *From ..lxml/src/lxml I run: * nosetests -v --pdb --nocapture Comparing with ElementTree 1.3.0 Comparing with cElementTree 1.3.0 TESTED VERSION: 3.5.0.beta1 Python: (major=2, minor=7, micro=10, releaselevel='final', serial=42) lxml.etree: (3, 5, 0, -99) libxml used: (2, 8, 0) libxml compiled: (2, 8, 0) libxslt used: (1, 1, 26) libxslt compiled: (1, 1, 26) lxml.html.tests.test_autolink.test_suite ... ok lxml.html.tests.test_basic.test_suite ... ok test_allow_tags (lxml.html.tests.test_clean.CleanerTest) ... ok test_clean_invalid_root_tag (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_excluded (lxml.html.tests.test_clean.CleanerTest) ... ok test_safe_attrs_included (lxml.html.tests.test_clean.CleanerTest) ... ok lxml.html.tests.test_clean.test_suite ... ok lxml.html.tests.test_diff.test_suite ... ok test_body (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... ok test_broken_attribute (lxml.html.tests.test_elementsoup.SoupParserTestCase) ... fish: Job 1, “nosetests -v --pdb --nocapture” terminated by signal SIGSEGV (Address boundary error) ``` Running these tests under gdb reveals the following partial stack trace: #0 0x00007ffff57fa08a in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #1 0x00007ffff57fa110 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #2 0x00007ffff5776be3 in PyPyWeakref_LockObject () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #3 0x00007ffff0337399 in __pyx_f_4lxml_5etree__isProxyAliveInPypy.isra.265 () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #4 0x00007ffff033fbfb in __pyx_f_4lxml_5etree_attemptDeallocation () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #5 0x00007ffff033fd70 in __pyx_tp_dealloc_4lxml_5etree__Element () from /home/jeff/lxml/src/lxml/etree.pypy-26.so #6 0x00007ffff62186f9 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #7 0x00007ffff57aec90 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #8 0x00007ffff57fa670 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #9 0x00007ffff5fa0da8 in ?? () from /home/jeff/debian_pypy_env/bin/libpypy-c.so #10 0x00007ffff5f89a31 in ?? ()

6 17

Leysin Sprint: instructions
by Armin Rigo 16 Feb '16

16 Feb '16

Hi all, If you're coming to the Leysin sprint, a few extra instructions: * look up your way on https://www.openstreetmap.org/#map=17/46.34727/7.01722 . Chalet Ermina is "up right" in this view. Scroll a bit left-down to see the train stations. The closest is Leysin-Feydey, though you may also go down in Versmont if you prefer less steep streets. It's around 10-15 minutes to walk. There is also a free bus touring the village hourly. * if you were there in previous years: the Chalet to go to is the "bottom" one. Go past the "top" one for 100m more, then back to the right 100m. * if you arrive Saturday before 5pm, meet me at L'Essentiel, a new restaurant nearby. If you arrive afterwards, go to the Chalet directly. * L'Essentiel is in the same place as an old restaurant which I'll not name here: google maps is not up-to-date, so it has only the old restaurant --- but it is misplaced... A bientôt, Armin.

2 2

What is the last good build of Pypy?
by Robert Grosse 15 Feb '16

15 Feb '16

I am trying to update Pypy. I want to build Pypy from source using the instructions at http://doc.pypy.org/en/latest/build.html in order to get the latest bugfixes and improvements. However, all of the last 5 nightly builds shown at http://buildbot.pypy.org/summary?branch=%3Ctrunk%3E failed their tests. So I can't tell what a good revision to update to is. What is the last good revision of Pypy?

4 12

s390x jit backend the first results and status update
by Richard Plangger 11 Feb '16

11 Feb '16

Hi, I have been looking quite elaborately to find the last remaining issues in the JIT backend. I identified some problems (related to the ABI for floats and ffi issues). Those are fixed. There are two major left: 1) Bug that occurs very late in the translation of PyPy. As you are aware I'm not able to reproduce it with the ssh access to my s390x machine. It looks to me like a double free of a memory chunk, or a miscalculation that only occurs when a lot of memory is allocated >4GB. When I can my hands on a 8GB RAM machine I'm sure I can fix it easily. 2) Speed is currently faster than CPython on all benchmarks but probably not on translation (which I could not run) and bm_mdp. See here [1]. If you compare it to the results reported by Armin for PPC & X86 ([2]) then I get around 50%-80% of the speedup. I have had a look at a lot of profiles (generated with valgrind) but did not find anything that is too slow. Would there be a difference when I run the benchmark on real hardware? Sadly not all tests on build bot are green. The ones that fail, fail because some thread did not start in time. Or some lock was hold too long. Eventually these tests will pass, but the suite just stops them. I cannot reproduce that on my s390x VM either, because it seems this only occurs under heavy load while running the test suite. At the end I want to mention that it seems that there is a problem related to the some special macros expanded by GCC (e.g. WCOREDUMP). I have reported the problem to the glibc bug tracker [3]. I have fixed the problem in pypy by altering the type provided to the macro, but there might be more problems while compiling pypy. Cheers, Richard [1] https://bpaste.net/show/d1af45831627 [2] http://morepypy.blogspot.co.at/2015/10/powerpc-backend-for-jit.html [3] https://sourceware.org/bugzilla/show_bug.cgi?id=19613

1 0

s390x backend gil and threading issue
by Richard Plangger 09 Feb '16

09 Feb '16

hi, I'm currently searching for a problem, I have debugged for quite a long time. I think this is the root problem why the pypy translation with pypy is still slower than cpython. Here are some of my findings (+questions): The list of last tests that fail all have one thing in common: They have an issue with the gil/threading. (See [1]) Most interesting ones are the last five. * test_gc_locking (2x) fail on the build bot (only using cpython), but not on my machine. This is strange because bbot and my vm use the same distro, same compiler version, .... the only difference is that bbot has better hardware and the tests are run with testrunner. Is there another way I can reproduce this? * test_ping_pong. (-A test) ping pong from one thread to another stressing locking and the GIL switch. On s390x and the translated VM this takes really long (10 seconds and on bbot it seems to exceed 30 seconds when run in parallel). However if I run the same test with PYPYLOG=jit:- it completes in ~0.96 seconds (in gdb it is the same). If you subtract the time needed for printing you might end up with the same speed as x86 has for this test. What does the printing/gdbing trigger to let the GIL switch happen that smoothly? * I have placed memory fences at the same positions as on ppc (2x isync and lwsync). Are there any other places that need to complete all pending the memory operations? * There is one path in call_release_gil (just after the call) where rpy_fastgil was acquired (because it as 0) and the shadowstack is not the one of the current thread. Then *rpy_fastgil = 0 is set for the slowpath function. Wouldn't it be possible to steal the gil at this point? Would that lead to a problem? Cheers, Richard [1] http://buildbot.pypy.org/summary?builder=own-linux-s390x

2 1

Question about "__extend__" hacks (in pypyjit module)
by Magnus Morton 09 Feb '16

09 Feb '16

Hi, In the __init__.py for the ‘pypyjit’ module there’s a comment in the ‘setup_after_space_initialization’ method "force the __extend__ hacks to occur early” (line 34) followed by two imports. What are these __extend__ hacks? Many thanks, Magnus

3 2