Mailman 3 July 2017 - pypy-dev

Error while integrating PyPy module in Yocto (Jethro branch) build.
by JAYMIN DABHI July 20, 2017

July 20, 2017

Hi All, I am trying to integrate PyPy in my current Yocto (Jethro branch) build for my i.MX6UL processor based development board. The board has "armv7a" (cortexa7) architecture. I am using "meta-pypy" layer from here (https://github.com/mzakharo/meta-pypy (https://github.com/mzakharo/meta-pypy)) and inherit the pypy package in my local.conf. But, it gives me error during bitbaking. I am bitbaking the "core-image-base". The error is as below : ERROR: Fetcher failure: Unable to find file file://pypy-4.0.1-cortexa7.tar.bz2 (file://pypy-4.0.1-cortexa7.tar.bz2/) anywhere. I think, it can't find the package "pypy-4.0.1-cortexa7.tar.bz2" in pypy directory. What should be the other possible reason? and How it can be solved? Suggestions are welcome. Thanks, - JAYMIN

2 1

(no subject)
by JAYMIN DABHI July 20, 2017

July 20, 2017

Hi All, I am trying to integrate PyPy in my current Yocto (Jethro branch) build for my i.MX6UL processor based development board. The board has "armv7a" (cortexa7) architecture. I am using "meta-pypy" layer from here (https://github.com/mzakharo/meta-pypy (https://github.com/mzakharo/meta-pypy)) and inherit the pypy package in my local.conf. But, it gives me error during bitbaking. I am bitbaking the "core-image-base". The error is as below : ERROR: Fetcher failure: Unable to find file file://pypy-4.0.1-cortexa7.tar.bz2 (file://pypy-4.0.1-cortexa7.tar.bz2/) anywhere. I think, it can't find the package "pypy-4.0.1-cortexa7.tar.bz2" in pypy directory. What should be the other possible reason? and How it can be solved? Suggestions are welcome. Thanks, -JAYMIN

1 0

cpyext performance
by Antonio Cuni July 19, 2017

July 19, 2017

Hello, recently I have been playing a bit with cpyext, so see if there are low haning fruits to be taken to improve the performance. I didn't get any real result but I think it's interesting to share my findings. The benchmark I'm using is here: https://github.com/antocuni/cpyext-benchmarks it contains a simple C extension defining three methods, one for each METH_NOARGS, METH_O and METH_VARARGS flags. So first, the results with CPython and PyPy 5.8: $ python bench.py noargs : 0.78 secs onearg : 0.89 secs varargs: 1.05 secs $ pypy bench.py noargs : 1.67 secs onearg : 2.13 secs varargs: 4.89 secs Then, I tried my cpyext-jit branch; this branch does two things: 1) it makes cpyext visible to the JIT, and add enough @jit.dont_look_inside so that it actually compiles 2) merges part of the cpyext-callopt branch, up to rev 9cbc8bd76297 (more on this later): this adds fast paths for METH_NOARGS and METH_O to avoid going through the slow __args__.unpack(): $ pypy-cpyext-jit bench.py noargs : 0.30 secs onearg : 0.31 secs varargs: 4.90 secs So, apparently this is enough to greatly speedup the calls, and be even faster than CPython. Note that "onearg" calls "simple.onearg(None)". However, things become more complicated as soon as I start passing various kind of objects to onearg(): $ pypy bench_oneargs.py # pypy 5.8 onearg(None): 2.09 secs onearg(1) : 2.07 secs onearg(i) : 4.98 secs onearg(i%2) : 4.92 secs onearg(X) : 2.13 secs onearg((1,)): 2.30 secs onearg((i,)): 9.80 secs $ pypy-cpyext-jit bench_oneargs.py onearg(None): 0.30 secs onearg(1) : 0.30 secs onearg(i) : 2.52 secs onearg(i%2) : 2.56 secs onearg(X) : 0.30 secs onearg((1,)): 0.30 secs onearg((i,)): 7.45 secs so, the call optimization still helps, but as soon as we need to convert one object from pypy to cpython we are horribly slow. However, it is interesting to note that: 1) if we pass a constant object, we are fast: None, 1, (1,) 2) if we pass X (which is a global X=100), we are still fast 3) any other object which is created on the fly is slow Looking at the traces, they look more or less the same in the three cases, so I don't really understand what is the difference. Finally, about the branch cpyext-callopt, which was started in Leysin by Richard, Armin and me: I am not sure to fully understand the purpose of dbba78b270fd: apparently, the optimization done in 9cbc8bd76297 seems to work well, so what am I missing? ciao, Anto

1 0

Call rpython from outside
by Aleksandr Koshkin July 16, 2017

July 16, 2017

Sup, guys. I want my rpython function to be invokable from outside world specifically be python. I have wrapped my function with entrypoint_highlevel and it appeared in shared object. So far so good. As a first argument this function takes a pointer to a C struct, and there is a problem. I have precisely recreated this struct in RPython as a lltypes.Struct (not rffi.CStruct) and annotated by this object my entrypoint signature, but it seams that some fields of the passed struct are messed up (shifted basically). Could it be because I used Struct instead of CStruct? I am using CFFI as a binding generator. -- Kind regards, Aleksandr Koshkin.

3 9

Re: [pypy-dev] Static bytecode instruction optimization and pypy (JIT?) optimization
by rymg19＠gmail.com July 11, 2017

July 11, 2017

Part of the problem is behavioral changes. If you implement tail call recursion, your tracebacks change. Common subexpression elimination will also have subtle behavioral changes. -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttp://refi64.com On Jul 9, 2017 at 8:11 PM, <Rocky Bernstein <rb(a)dustyfeet.com>> wrote: In looking at Python bytecode over the years, I've noticed that it does very little to no traditional static-compiler optimization. Specifically: * Dead code elmination (most of the time) * Jumps to Jumps or Jumps to the next instruction * Constant propagation (most of the time) * Common subexpression elimination * Tail recursion * Code hoisting/sinking Yes, over the years more compiler optimization has been done but it's still pretty sparse. The little that I've looked at pypy, it is pretty much the same thing at the Python bytecode level. That's why a python decompiler for say 2.7 will work largely unmodified for he corresponding pypy 2.7 version. Same is true 3.2 versus pypy 3.2. I understand though that pypy can do and probably does some of the optimization when it JITs. But my question is: if traditional compiler optimization were done would this hinder, be neutral, or help pypy's optimization. Of course, if there were static compiler optimization of the type described above, that might be a win when JITing doesn't kick in. (And perhaps then who cares) But I'm interested in the other situation where both are in play. _______________________________________________ pypy-dev mailing list pypy-dev(a)python.org https://mail.python.org/mailman/listinfo/pypy-dev

5 5

Re: [pypy-dev] Static bytecode instruction optimization and pypy (JIT?) optimization
by William ML Leslie July 11, 2017

July 11, 2017

On 11 July 2017 at 18:22, Rocky Bernstein <rb(a)dustyfeet.com> wrote: > There's too much generalization and abstraction in the discussion at least > for me. So let me try to make this a little more concrete with real > examples. First note that, although Python, yes, does "optimize" "if 0:" > away, as soon as > you do something as simple as "debug = 0; if debug:" it is confounded. And > in my own experience I have done this. > Typically DEBUG is a global in the module, which means it can be set from /outside/ the module, so it's not constant as far as the runtime is concerned. Within a function it's a bit wacky sure, and is only kept around for interactive debugging reasons afaict. Here's a possible type for `self.a` which illustrates why CSE/Store-Sinking on a STORE_ATTR is not valid for the example function `foo`. class A(object): def set_b(self, value): print value self._b = value b = property(set_b) class Example(object): def __init__(self): self.a = A() def foo(self): self.a.b = 5 x = 0 if x: self.a.b = 2 else: return Example().foo() -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.

2 2

Static bytecode instruction optimization and pypy (JIT?) optimization
by Rocky Bernstein July 11, 2017

July 11, 2017

In looking at Python bytecode over the years, I've noticed that it does very little to no traditional static-compiler optimization. Specifically: * Dead code elmination (most of the time) * Jumps to Jumps or Jumps to the next instruction * Constant propagation (most of the time) * Common subexpression elimination * Tail recursion * Code hoisting/sinking Yes, over the years more compiler optimization has been done but it's still pretty sparse. The little that I've looked at pypy, it is pretty much the same thing at the Python bytecode level. That's why a python decompiler for say 2.7 will work largely unmodified for he corresponding pypy 2.7 version. Same is true 3.2 versus pypy 3.2. I understand though that pypy can do and probably does some of the optimization when it JITs. But my question is: if traditional compiler optimization were done would this hinder, be neutral, or help pypy's optimization. Of course, if there were static compiler optimization of the type described above, that might be a win when JITing doesn't kick in. (And perhaps then who cares) But I'm interested in the other situation where both are in play.

2 1

Emulate pypy SIGSEGV
by Pavlo Lavrenenko July 10, 2017

July 10, 2017

Hello, guys. Running PyPy 5.6.0 with GCC 4.8.5 20150623 (Red Hat 4.8.5-4) here. Occasionally PyPy terminates with signal 11 and a long stack consisting mostly of: #31 0x00007f0d4c26ce7d in ?? () from /lib64/libpypy-c.so If I had a proper code sample to reproduce the issue I would have submitted it but since I am literally chasing ghosts I am more interested in how to make PyPy throw SIGSEGV on purpose (the software hangs after one of its child processes dies with signal 11, I want to at least reproduce and workaround this). kill -11 doesn't give the result I want. Any way to make PyPy crash somewhere in /lib64/libpypy-c.so? -- Best regards, Pavlo Lavrenenko, PortaOne, Inc., Junior Software Developer Tel: +1-866-SIP VOIP (+1 866 747 8647) ext. 7624 PortaOne - VoIP Solutions Company Visit our Website: http://www.portaone.com

3 5

Re: [pypy-dev] Asking for help
by William ML Leslie July 3, 2017

July 3, 2017

On 3 July 2017 at 15:47, Meide Zhao <zhao.meide(a)gmail.com> wrote: > Thanks a lot Leslie. > > Can I use the newer version of Ubuntu like 16.04 or 17.04? Which Linux and > which version do you use? > Tannit (which does the automatic linux-64 builds) has GCC 4.8.2, so even something that old should work. I'm not sure if it differs from 12.04 in any other way. I build pypy regularly on Ubuntu 16.04 LTS (has gcc 5.4.0) as well as Debian Jessie (with gcc 4.9.2). -- William Leslie Notice: Likely much of this email is, by the nature of copyright, covered under copyright law. You absolutely MAY reproduce any part of it in accordance with the copyright law of the nation you are reading this in. Any attempt to DENY YOU THOSE RIGHTS would be illegal without prior contractual agreement.

2 1

Asking for help
by Meide Zhao July 3, 2017

July 3, 2017

Hi all, I'm trying to build pypy from source on ubuntu 12.04 LTS but can't get it to work (see error messages below). Can someone help me and let me know the correct steps since I'm new to pypy? Thanks, Meide (1) The details for the ubuntu in virtualbox are as follows cat /etc/*-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=12.04 DISTRIB_CODENAME=precise DISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS" NAME="Ubuntu" VERSION="12.04.3 LTS, Precise Pangolin" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu precise (12.04.3 LTS)" VERSION_ID="12.04" uname -a Linux ubuntu 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 16:19:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux python -V Python 2.7.13 (2) install pypy dependencies via command: sudo apt-get install gcc make libffi-dev pkg-config zlib1g-dev libbz2-dev libsqlite3-dev libncurses5-dev libexpat1-dev libssl-dev libgdbm-dev tk-dev libgc-dev liblzma-dev mercurial (3) check out pypy source code: cd ~/pypy hg clone http://bitbucket.org/pypy/pypy pypy (4)build pypy cd ~/pypy/pypy/pypy/goal/ sudo ../../rpython/bin/rpython --opt=jit targetpypystandalone.py [platform:execute] gcc -c -O3 -pthread -fomit-frame-pointer -Wall -Wno-unused -Wno-address -DRPYTHON_VMPROF -O3 -DVMPROF_UNIX -DVMPROF_LINUX -DRPYTHON_LL2CTYPES -I/home/meide/pypy/pypy/rpython/rlib/rvmprof/src -I/home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared -I/home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c -o /tmp/usession-default-0/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.o [translation:info] Error: File "/home/meide/pypy/pypy/rpython/translator/goal/translate.py", line 284, in main default_goal='compile') File "/home/meide/pypy/pypy/rpython/translator/driver.py", line 566, in from_targetspec spec = target(driver, args) File "targetpypystandalone.py", line 337, in target return self.get_entry_point(config) File "targetpypystandalone.py", line 368, in get_entry_point space = make_objspace(config) File "/home/meide/pypy/pypy/pypy/tool/option.py", line 35, in make_objspace return Space(config) File "/home/meide/pypy/pypy/pypy/interpreter/baseobjspace.py", line 441, in __init__ self.initialize() File "/home/meide/pypy/pypy/pypy/objspace/std/objspace.py", line 105, in initialize self.make_builtins() File "/home/meide/pypy/pypy/pypy/interpreter/baseobjspace.py", line 637, in make_builtins self.install_mixedmodule(mixedname, installed_builtin_modules) File "/home/meide/pypy/pypy/pypy/interpreter/baseobjspace.py", line 668, in install_mixedmodule modname = self.setbuiltinmodule(mixedname) File "/home/meide/pypy/pypy/pypy/interpreter/baseobjspace.py", line 507, in setbuiltinmodule None, None, ["Module"]).Module File "/home/meide/pypy/pypy/pypy/module/_vmprof/__init__.py", line 30, in <module> import pypy.module._vmprof.interp_vmprof File "/home/meide/pypy/pypy/pypy/module/_vmprof/interp_vmprof.py", line 14, in <module> my_execute_frame = _decorator(PyFrame.execute_frame) File "/home/meide/pypy/pypy/rpython/rlib/rvmprof/rvmprof.py", line 196, in decorate _get_vmprof() File "/home/meide/pypy/pypy/rpython/rlib/rvmprof/rvmprof.py", line 243, in _get_vmprof _vmprof_instance = VMProf() File "/home/meide/pypy/pypy/rpython/rlib/rvmprof/rvmprof.py", line 49, in __init__ self.cintf = cintf.setup() File "/home/meide/pypy/pypy/rpython/rlib/rvmprof/cintf.py", line 82, in setup **eci_kwds)) File "/home/meide/pypy/pypy/rpython/rtyper/tool/rffi_platform.py", line 94, in verify_eci configure(CConfig) File "/home/meide/pypy/pypy/rpython/rtyper/tool/rffi_platform.py", line 223, in configure res[key] = value.question(writer.ask_gcc) File "/home/meide/pypy/pypy/rpython/rtyper/tool/rffi_platform.py", line 555, in question ask_gcc("") File "/home/meide/pypy/pypy/rpython/rtyper/tool/rffi_platform.py", line 191, in ask_gcc try_compile_cache([self.path], self.eci) File "/home/meide/pypy/pypy/rpython/tool/gcc_cache.py", line 71, in try_compile_cache platform.compile(c_files, eci) File "/home/meide/pypy/pypy/rpython/translator/platform/__init__.py", line 54, in compile ofiles = self._compile_o_files(cfiles, eci, standalone) File "/home/meide/pypy/pypy/rpython/translator/platform/__init__.py", line 76, in _compile_o_files ofiles.append(self._compile_c_file(self.cc, cfile, compile_args)) File "/home/meide/pypy/pypy/rpython/translator/platform/posix.py", line 41, in _compile_c_file cwd=str(cfile.dirpath())) File "/home/meide/pypy/pypy/rpython/translator/platform/__init__.py", line 140, in _execute_c_compiler self._handle_error(returncode, stdout, stderr, outname) File "/home/meide/pypy/pypy/rpython/translator/platform/__init__.py", line 152, in _handle_error raise CompilationError(stdout, stderr) [translation:ERROR] CompilationError: CompilationError(err=""" /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c: In function ‘elf_add_syminfo_data’: /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:448:8: warning: implicit declaration of function ‘__atomic_load_n’ [-Wimplicit-function-declaration] /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:448:12: error: ‘__ATOMIC_ACQUIRE’ undeclared (first use in this function) /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:448:12: note: each undeclared identifier is reported only once for each function it appears in /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c: In function ‘elf_syminfo’: /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:493:12: error: ‘__ATOMIC_ACQUIRE’ undeclared (first use in this function) /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c: In function ‘backtrace_initialize’: /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:958:2: warning: implicit declaration of function ‘__atomic_store_n’ [-Wimplicit-function-declaration] /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:958:2: error: ‘__ATOMIC_RELEASE’ undeclared (first use in this function) /home/meide/pypy/pypy/rpython/rlib/rvmprof/src/shared/libbacktrace/elf.c:973:20: error: ‘__ATOMIC_ACQUIRE’ undeclared (first use in this function) """) [translation] start debugger... > /home/meide/pypy/pypy/rpython/translator/platform/__init__.py(152)_handle_error() -> raise CompilationError(stdout, stderr) (Pdb+)

3 3