Mailman 3 May 2016 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

Update to the win32 buildbot
by Matti Picus 10 Jul '16

10 Jul '16

Hi! I am cc-ing the pypy-dev list so there is some permanent record of this, the mail is for the win32 buildbot maintainer. Could you add the lzma development libraries to the build environment of the win32 buildbot? They are needed for python 3, as you can see from any of the failing build runs like this one http://buildbot.pypy.org/builders/pypy-c-jit-win-x86-32/builds/2398/steps/s… Python 3 uses the xz-5.0.5 libraries, as can bee seen in the cpython/PCbuild/readme.txt, following their links leads to http://tukaani.org/xz/ and to a download http://tukaani.org/xz/xz-5.0.5-windows.zip (library) http://tukaani.org/xz/xz-5.0.5-windows.zip.sig (signature) I will try to update the local.zip available on our downloads https://bitbucket.org/pypy/pypy/downloads/local_2.4.zip and the documentation http://doc.pypy.org/en/latest/windows.html but in the meantime it would be great if you could get this going Thanks! Matti

2 1

pypy3.3-v5.2-alpha1 release
by Matti Picus 05 Jun '16

05 Jun '16

We are almost ready to release pypy3.3-v5.2-alpha1, but need some help to make sure all is OK. Please try the download packages here https://bitbucket.org/pypy/pypy/downloads also, if someone could check that the links related to pypy3.3 all work on this page that would be great http://pypy.org/download.html If the page is still showing PyPy3 2.4.0, wait 30 minutes or so from now and then hit refresh on your browser. Thanks, and kudos to the pypy3 team. Matti

4 4

asmgcc versus shadowstack
by Armin Rigo 30 May '16

30 May '16

Hi all, Recently, we've got a few more of the common bug report "cannot find gc roots!". This is caused by asmgcc somehow failing to parse the ".s" files produced by gcc on Linux. I'm investigating what can be done to improve the situation of asmgcc in a more definitive way. There are basically two solutions: 1) we improve shadowstack. This is the alternative to asmgcc, which is used on any non-Linux platform already. So far it is around 10% slower than asmgcc. 2) we improve asmgcc by finding some better way than parsing assembler files. I worked during the past month in the branch "shadowstack-perf-2". This gives a major improvement on the placement of pushing and popping GC roots on the shadow stack. I think it's worth merging that branch in any case. On x86, it gives roughly 3-4% speed improvements; I'd guess on arm it is slightly more. (I'm comparing the performance outside JITted machine code; the JITted machine code we produce is more similar.) The problem is that asmgcc used to be ~10% better. IMHO, 3-4% is not quite enough to be happy and kill asmgcc. Improving beyond these 3-4% seems to require some new ideas. So I'm also thinking about ways to fix asmgcc more generally, this time focusing on Linux only; asmgcc contains old code that tries to parse MSVC output, and I bet we tried with clang at some point, but these attempts both failed. So let's focus on Linux and gcc only. Asmgcc does two things with the parsed assembler: it computes the stack size at every point, and it tracks some marked variables backward until the previous "call" instruction. I think we can assume that the version of gcc is not older than, say, the one on tannit32 (Ubuntu 12.04), which is gcc 4.6. At least from that version, both on x86-32 and x86-64, gcc will emit "CFI directives" (https://sourceware.org/binutils/docs/as/CFI-directives.html). These are a saner way to get the information about the current stack size. About the backward tracking, we need to have a complete understanding of all instructions, even if e.g. for any xmm instruction we just say "can't handle GC pointers". The backward tracking itself is often foiled because the assembler is lacking a way to know clearly "this call never returns" (e.g. calls to abort(), or to some RPython helper that prints stuff and aborts). In other words, the control flow is sometimes hard to get correctly, because a "call" generally returns, but not always. Such mistakes can produce bogus results (including "cannot find gc roots!"). What can we do about that? Maybe we can compile with "-s -fdump-final-insns". This dumps a gcc-specific summary of the RTL, which is the final intermediate representation, which looks like it is in one-to-one correspondance with the actual assembly. It would be a better input for the backward-tracker, because we don't have to handle tons of instructions with unknown effects, and because it contains explicit points at which control flow cannot pass. On the other hand, we'd need to parse both the .s and this dump in parallel, matching them as we go along. But I still think it would be better than now. Of course the best would be to get rid of asmgcc completely... This mail is meant to be a dump of my current mind's state :-) A bientôt, Armin.

2 1

Looking into numpy ndarray.flags.writeable
by Eli Stevens (Gmail) 29 May '16

29 May '16

The following works on cpython, but fails on pypy: import numpy as np def test_writeable(): a = np.zeros((2,2), np.int8) > a.flags.writeable = False E TypeError: readonly attribute So I went and installed pypy's fork of numpy from source, and tried to start poking around looking for tests, etc. to see what was going on. It looks like numpy/core/tests/test_multiarray.py TestFlags.test_writeable is doing something similar, but when I attempt to run the tests like so: pypy -c "import numpy; numpy.test()" (Is that the right way to do it?) It looks like the only thing that gets referenced from the test_multiarray.py file is: ====================================================================== ERROR: Failure: AttributeError ('module' object has no attribute 'datetime64') ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/elis/venv/droidblue-pypy/site-packages/nose/loader.py", line 418, in loadTestsFromName addr.filename, addr.module) File "/Users/elis/venv/droidblue-pypy/site-packages/nose/importer.py", line 47, in importFromPath return self.importFromDir(dir_path, fqname) File "/Users/elis/venv/droidblue-pypy/site-packages/nose/importer.py", line 94, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/Users/elis/venv/droidblue-pypy/site-packages/numpy/core/tests/test_multiarray.py", line 2712, in <module> class TestArgmax(TestCase): File "/Users/elis/venv/droidblue-pypy/site-packages/numpy/core/tests/test_multiarray.py", line 2733, in TestArgmax ([np.datetime64('1923-04-14T12:43:12'), AttributeError: 'module' object has no attribute 'datetime64' (And this is after I have to comment out a bunch of imports that can't be found, etc.) My reading of this is that none of the tests in the test_multiarray.py file are even being attempted due to a missing datetime64. Is that correct? Since we're not supposed to touch the numpy source if at all possible, what's the suggested approach for getting the TestFlags tests to a state where I can run them (besides just hacking away at the source file, as it seems unlikely that would be an acceptable patch)? I'm not certain if I'm going to be able to invest the time to fix the writeable flag, but I'd like to give it a go, at least. Anything else in particular I should be aware of, when it comes to flags, etc.? Thanks, Eli

6 16

Question about cookielib in PyPy
by Jeff Doran 28 May '16

28 May '16

I'm working on an ongoing task to test our existing code on PyPy. My latest exploration was is using PyPy 5.1 along with lxml 3.6.0 and of partuclar relevance to my question, requests 2.9.1 and betamax 0.7.0 Our test suite completes without incident on Python 2.7.9, but on PyPy 5.1.0 we encounter errors trying to match an http request to a recorded session in betamax. This process fails when trying to match a request against the correct recorded session due to a failure in the header comparison when we include a cookie string. This is a comparison of 2 dicts and the contents are the same, except for the Cookie string. We created the Cookie using cookielib.Cookie and add the various items via the Cookie.__init__() method. The resulting strings comes from Cookie.__str__ I believe and the comparison in requests is a simple dict1 == dict2. Any thoughts on where this difference might lie (dictionary comparison issues, string comparison issues, or ...)? I'd like to avoid having to have a different set of tests for PyPy vs CPython as that defeats the idea of verifying compatibility. Thanks for any input. - Jeff Doran This comes from cookielib.Cookie under PyPY. request.headers: %s { 'User-Agent': 'python-requests/2.9.1', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Cookie': 'NextGenCSO=8MWACYK5f24yllbpMwVpocfsyGYIBUXl4aWQvo7nUcWz2OwU7p4Dy40bxiyjGS5es8hBv5LxPT8PqBnWNzaBNo0k0PGffpQTDI4xBGc9WwQevnzyUCmq7WaXMTOTSpKM; PacerClientCode=dev; PacerSession=8MWACYK5f24yllbpMwVpocfsyGYIBUXl4aWQvo7nUcWz2OwU7p4Dy40bxiyjGS5es8hBv5LxPT8PqBnWNzaBNo0k0PGffpQTDI4xBGc9WwQevnzyUCmq7WaXMTOTSpKM; domain=.uscourts.gov; path=/; PacerPref="receipt=Y"' } This was recorded previously using Python 2.7.9 recorded headers: %s { u'Connection': u'keep-alive', u'Cookie': u'PacerClientCode=dev; path=/; NextGenCSO=8MWACYK5f24yllbpMwVpocfsyGYIBUXl4aWQvo7nUcWz2OwU7p4Dy40bxiyjGS5es8hBv5LxPT8PqBnWNzaBNo0k0PGffpQTDI4xBGc9WwQevnzyUCmq7WaXMTOTSpKM; domain=.uscourts.gov; PacerSession=8MWACYK5f24yllbpMwVpocfsyGYIBUXl4aWQvo7nUcWz2OwU7p4Dy40bxiyjGS5es8hBv5LxPT8PqBnWNzaBNo0k0PGffpQTDI4xBGc9WwQevnzyUCmq7WaXMTOTSpKM; PacerPref="receipt=Y"', u'Accept-Encoding': u'gzip, deflate', u'Accept': u'*/*', u'User-Agent': u'python-requests/2.9.1' }

2 1

Found on the internet
by Maciej Fijalkowski 27 May '16

27 May '16

Another game boy emulator using RPython https://github.com/Baekalfen/PyBoy

2 1

Forwarding...
by Armin Rigo 23 May '16

23 May '16

On 19 May 2016 at 14:58, <pypy-dev-owner(a)python.org> wrote: > ---------- Forwarded message ---------- > From: Daniel Hnyk <hnykda(a)gmail.com> > To: pypy-dev(a)python.org > Cc: > Date: Thu, 19 May 2016 12:58:36 +0000 > Subject: Question about funding, again > Hello, > > my question is simple. It strikes me why you don't have more financial support, since PyPy might save quite a lot of resources compared to CPython. When we witness that e.g. microsoft is able to donate $100k to Jupyter (https://ipython.org/microsoft-donation-2013.html), why PyPy, being even more generic then Jupyter, has problem to raise few tenths of thousands. > > I can find few mentions about this on the internet, but no serious article or summary is out there. > > Have you tried any of the following? > > 1. Trying to get some funding from big companies and organizations such as Google, Microsoft, RedHat or some other like Free Software Foundation? If not, why not? > 2. Crowd founding websites such as Kickstarter or Indiegogo get quite a big attention nowadays even for similar projects. There were successful campaigns for projects with even smaller target group, such as designers (https://krita.org/) or video editors (openshot 2). Why haven't you created a campaign there? Micropython, again, with much smaller target group of users had got funded as well. > > Is someone working on this subject? Or is there a general lack of man power in PyPy's team? Couldn't be someone hired from money already collected? > > Thanks for an answer, > Daniel

4 6

Question about funding, again
by Daniel Hnyk 19 May '16

19 May '16

Hello, my question is simple. It strikes me why you don't have more financial support, since PyPy might save quite a lot of resources compared to CPython. When we witness that e.g. microsoft is able to donate $100k to Jupyter (https://ipython.org/microsoft-donation-2013.html), why PyPy, being even more generic then Jupyter, has problem to raise few tenths of thousands. I can find few mentions about this on the internet, but no serious article or summary is out there. Have you tried any of the following? 1. Trying to get some funding from big companies and organizations such as Google, Microsoft, RedHat or some other like Free Software Foundation? If not, why not? 2. Crowd founding websites such as Kickstarter or Indiegogo get quite a big attention nowadays even for similar projects. There were successful campaigns for projects with even smaller target group, such as designers ( https://krita.org/) or video editors (openshot 2). Why haven't you created a campaign there? Micropython, again, with much smaller target group of users had got funded as well. Is someone working on this subject? Or is there a general lack of man power in PyPy's team? Couldn't be someone hired from money already collected? Thanks for an answer, Daniel

1 0

RUMPLE'16 Call for Papers
by Carl Friedrich Bolz 17 May '16

17 May '16

============================================================================ Call for Papers: RUMPLE’16 1st Workshop on ReUsable and Modular Programming Language Ecosystems Co-located with SPLASH Oct/Nov, 2016, Amsterdam, Netherlands http://2016.splashcon.org/track/rumple2016 ============================================================================ The RUMPLE workshop is a venue for discussing modular approaches to programming language implementations, extensible virtual machine architectures, as well as reusable runtime components such as dynamic compilers, interpreters, or garbage collectors. The main goal of the workshop is to bring together researchers and practitioners, and facilitate the sharing of experiences and ideas. Relevant topics include, but are definitely not limited to, the following: - Extensible VM design (compiler- or interpreter-based VMs) - Reusable implementation of runtime components (e.g. interpreters, garbage collectors, intermediate representations) - Static and dynamic compiler techniques for different languages - Multi-language runtimes and mechanisms for cross-language interoperability between different languages - Tooling support for different languages (e.g. debugging, profiling, etc.) - Modular language implementations that use existing frameworks and systems - Case studies of existing language implementations, virtual machines, and runtime components (e.g. design choices, tradeoffs, etc.) - New research ideas on how we want to build languages in the future. Workshop Format and Submissions This workshop welcomes the presentation and discussion of new ideas and emerging problems that give a chance for interaction and exchange. We accept presentation proposals in the form of extended abstracts (1-4 pages). Accepted abstracts will be published on the workshop's website before the workshop date. Submissions should use the ACM SIGPLAN Conference Format, 10 point font, using the font family Times New Roman and numeric citation style. All submissions should be in PDF format. Please submit abstracts through http://ssw.jku.at/rumple/ Important Dates - Exended abstract submission: 1 Aug 2016 - Author notification: 5 Sep 2016 All deadlines are Anywhere on Earth (AoE), i.e. GMT/UTC−12:00 hour - Workshop: 31 Oct 2016 Program Committee Walter Binder, University of Lugano Carl Friedrich Bolz, King's College London Richard Jones, University of Kent Stephen Kell, University of Cambridge Jan Vitek, Northeastern University Christian Wimmer, Oracle Labs Workshop Organizers Matthias Grimmer, Johannes Kepler University Linz, Austria Laurence Tratt, King's College London, United Kingdom Adam Welc, Oracle Labs, United States For questions or concerns, please mail to matthias.grimmer at jku.at

1 0