Mailman 3 April 2017 - pypy-dev

About gurobipy
by Luis José Novoa 31 May '18

31 May '18

Hi All, My name is Luis Novoa. Im working on a large scale optimization problem using python/gurobipy for my implementation. I was wondering if any of you in this list has used gurobipy with pypy, or if it is even possible at this time. Thanks in advance, Luis.

9 9

win32 buildbot
by Matti Picus 30 Apr '17

30 Apr '17

Any volunteers to host a win32 buildbot slave? It would be nice to have at least two machines online, since the processes seem so slow on that platform. The current builder has gotten stuck and the maintainer is not responding to emails, we are down to no functioning buildbot slaves for that platform. Minimum requirements are: - relatively new hardware - at least 8GB memory - 5 GB disk space for pypy (not including OS and visual studio) - internet connection I looked into an Azure vm, but from what I could tell it would be about $10 per translation, and maybe more for an own-tests run, since the machines with significant RAM are expensive. We tried once to get Microsoft to donate some credits, but that did not go far. Matti

2 7

Sigsegv from Cython
by Tin Tvrtković 21 Apr '17

21 Apr '17

Hello, I'm working on a feature for the attrs library. As part of that feature I need to wrap the member_descriptor descriptor from ordinary slot classes. In order to speed this up, I'm using Cython. I'm not really sure if this or the Cython mailing list is the right place to ask this, but since I've had better luck here in the past I'm posting this here first :) A quick reproducer: Ubuntu 17.04/amd64, make a PyPy 5.7.1 venv, pip install Cython, download https://gist.github.com/Tinche/678a2f97f656421ab65338ea678df354 and https://gist.github.com/Tinche/111ccd2f46c9ba303b8e441d2e0fcddb. Uncomment the last line in t.py to trigger the crash. This code seems to work on CPython (and it speeds up attribute access a lot there). I'm not even sure doing this on PyPy is worthwhile, but as long as it crashes I can't be sure :) I'll be the first to admit I don't have a lot of experience writing Cython or C, so maybe there's an amateur mistake in there. For example, I tried putting in a few Py_INCREFs to maybe stop PyPy from moving objects around but it doesn't seem to help. Thanks in advance: :)

2 1

Cross-translating for ARM on Ubuntu 16.04
by Zdenek Bouska 20 Apr '17

20 Apr '17

Hi, I would like to cross-translate pypy for armhf (Beagle Bone Black) on amd64 hardware running Ubuntu 16.04. I tried it using http://doc.pypy.org/en/release-2.4.x/arm.html (linked from but failed because Ubuntu 16.04 does not have scratchbox2 package. Scratchbox2 is no longer maintained according to Wikipedia. Does anyone here successfully cross-translated PyPy without scratchbox2? Regards Zdenek

1 0

PyPy Benchmarks
by Frank Wang 19 Apr '17

19 Apr '17

Hi, I was wondering if there were a "standard" set of PyPy benchmarks that are commonly used. I'm currently using this: http://pyperformance.readthedocs.io/ Thanks, Frank

2 3

Socket Module
by Frank Wang 18 Apr '17

18 Apr '17

Hi, I want to modify the definition of socket.socket() to include an extra parameter. I found this function (newsocket): https://bitbucket.org/pypy/pypy/src/8175c5e20480b2c8e1b95c7f758458db7193f1a… Is this the right function to modify? Since W_Socket inherits from W_Root, in the socket call, I want to call a function in W_Root. Should I call it on the space.wrap(self) object or just self? My question is more around that "self" seems to be a wrapped interpreter object already, specifically W_Socket. Why does it need to wrapped again? Unless I'm confused.... Frank

1 0

Tune performance benchmarks for PyPy
by Victor Stinner 18 Apr '17

18 Apr '17

Hi, I ran the performance benchmark suite with PyPy 5.7 on the speed-python server, the benchmark runner feeding speed.python.org. I chose to spawn 10 processes, each process computed 250 values. The number of outer loops was calibrated per benchmark. Here are raw data: http://www.haypocalc.com/tmp/pypy_p10_w0_n250.json.gz The plan is now to choose the number of warmup values from these data and copy the number of loops to get fixed parameters. You can directly analyze the (compressed) JSON file using plot.py to plot values: https://github.com/haypo/perf/raw/master/doc/examples/plot.py I suggest to use the --split-runs value to have one line per run (per process): python3 doc/examples/plot.py ~/pypy_p10_w0_n250.json.gz -b telco --split-runs It's likely that the first values are much larger: use the UI to zoom, or pass --skip=N to skip the first N values of each run. python3 doc/examples/plot.py ~/pypy_p10_w0_n250.json.gz -b telco --split-runs --skip=3 Or you can only render one specific run: python3 doc/examples/plot.py ~/pypy_p10_w0_n250.json.gz -b telco --split-runs --run=2 The JSON file contains results of 67 benchmarks: use "python3 -m perf show ~/pypy_p10_w0_n250.json.gz -q" to list them. plot.py script requires a benchmark name (-b/--benchmark option). The game is now to choose the number of values to skip to reach the "steady state". plot.py renders the mean to help you ;-) You can also use perf commands to analyze data: * perf stats * perf dump * perf hist * etc. These commands can display all benchmarks, or only one if you use the -b option. Or you can use perf API to export data and use whatever you want to analyze data. See also the perf doc: http://perf.readthedocs.io/en/latest/analyze.html Next week, I will try to do the work for each benchmark, and then I will come back to you with numbers, so we can double-check together that everything looks fine. Parameters are likely to change depending on the CPU, PyPy version, etc. But my first goal is to get reproductible results on speed-python with the latest version of PyPy, so I will start with that. Later, we can adjust parameters depending on the CPU and/or PyPy version. Enjoy! Victor

1 1

Visual Studio 2015 build strange behavior
by Sergey Kurdakov 11 Apr '17

11 Apr '17

Hi, I managed to build pypy with visual studio 2015 ( win32 build on window 10/64 ) with both python and by pypy itself but I have one strange behavior it manifest in a first steps using new pypy by running pip pypy-c -mpip install -U wheel and an error reads Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x042cd170>: Failed to establish a new connection: [Errno 10035] A non-blocking socket operation could not be completed immediately',)': /simple/whe exactly error is: [Errno 10035] A non-blocking socket operation could not be completed immediately' ( the same error [Errno 10035] A non-blocking socket operation could not be completed immediately', also appears if I try to use PyMySQL driver to connect to mysql , so this is not that something happened to pip repository) the reason I built with Visual Studio 2015 was that stock pypy for windows would crash on my projects - and I have visual studio 2015. the build I obtained also seems somewhat more stable to my eyes, so it worth the try. but now I have seemingly more stable pypy, but at the same time that network error, any hints what might be involved here and how to debug? Regards Sergey

4 16

PyPy3.5 on FreeBSD
by David Naylor 10 Apr '17

10 Apr '17

Hi A quick note: pypy3.5 is running on FreeBSD/amd64 (aka 64bit), at least well enough to compile the CFFI modules during packaging of pypy3. As noted in the release, pypy3 does not run from i386 (aka 32bit): # /wrkdirs/usr/ports/lang/pypy3/work/pypy3-v5.7.1-src/pypy/goal/pypy3-c 'import site' failed Python 3.5.3 (2875f328eae2216a87f3d6f335092832eb031f56, Apr 09 2017, 05:33:17) [PyPy 5.7.1-beta0 with GCC 4.2.1 Compatible FreeBSD Clang 3.8.0 (tags/RELEASE_380/final 262564)] on freebsd11 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``probably the last time in the season, but: snow, snow!'' TypeError: 'NoneType' has no length Regards David (on behalf of the FreeBSD/Python team)

3 2

performance benchmark suite
by Victor Stinner 07 Apr '17

07 Apr '17

Hi, I'm working on speed.python.org, CPython benchmark. I reworked the benchmark suite which is now called "performance": http://pyperformance.readthedocs.io/ performance contains 54 benchmarks and works on Python 2.7 and 3.x. It creates a virtual environment with pinned versions of requirements to "isolate" the benchmark from the system, to get more reproductible results. I added a few benchmarks from the PyPy benchmark suite but I didn't add all of them yet. performance is now based on my perf module. The perf module is a toolkit to run, analyze and compare benchmarks: http://perf.readthedocs.io/ I would like to know how to adapt perf and performance to handle correctly PyPy JIT compiler: I would like to measure the performance when code has been optimized by the JIT compiler and ignore the warmup phase. I already made a few changes in perf and performance when a JIT is detected, but I'm not sure that I did them correctly. My final goal would be to have PyPy benchmark results on speed.python.org, to easily compare CPython and PyPy (using the same benchmark runner, same physical server). The perf module calibrates a benchmark based on time: it computes the number of outer loops to get a timing of at least 100 ms. Basically, a single value is computed as: t0 = perf.perf_counter() for _ in range(loops): func() value = perf.perf_counter() - t0 perf spawn a process only to calibrate the benchmark. On PyPy, it now (in the master branch) spawns a second process only computing warmup samples to validate the calibration. If a value becomes less than 100 ms, it doubles each time the number of loops. The opereation is repeated until the number of loops doesn't change. After the calibration, perf spawns worker processes sequentially: each worker computes warmup samples and then compute values. By default, each worker computes 1 warmup sample and 3 samples on CPython, and 10 warmup samples an 10 samples on PyPy. The configuration for PyPy is kind of arbitrary, wheras it was finely tuned for CPython. At the end, perf ignores all warmup samples and only computes the mean and standard deviations of other values. For example, on CPython 21 processes are spawned: 1 calibration + 20 workers, each worker computes 1 warmup + 3 values: compute the mean of 60 values. perf stores all data in a JSON file: metadata (hostname, CPU speed, system load, etc.), number of loops, warmup samples, samples, etc. It provides an API to access all data. perf also contains a lot of tools to analyze data: statistics (min, max, median/MAD, percentiles, ...), render an histogram, compare results and check if the difference is significant, detect unstable benchmark, etc. perf also contains a documentation explaining how to: run benchmark, analyze benchmarks, get stable/reproductible results, tune your system to run a benchmark, etc. To tune your system for benchmarks, run the "sudo python3 -m perf system tune" command. It configures the CPU (disable Turbo Boost, set a fixed frequency, ...), check that the power cable is plugged, set CPU affinity on IRQs, disable Linux perf events, etc. The command reduces the operating system jitter. Victor

2 2