Mailman 3 January 2012 - pypy-dev

Python vs pypy: interesting performance difference [dict.setdefault]
by David Naylor 10 Mar '14

10 Mar '14

Hi, I needed to create a cache of date and time objects and I wondered what was the best way to handle the cache. For comparison I put together the following test: <file="iforkey.py"> import datetime import random import timeit ranges = [datetime.datetime(2011,01, random.randint(1, 31)) for i in xrange(1000)] def ifdict(): cache = {} b = [] for i in ranges: key = i.day if key in cache: b.append(cache[key]) else: date = i.date() cache[key] = date b.append(date) def keydict(): cache = {} b = [] for i in ranges: key = i.day try: b.append(cache[key]) except KeyError: date = i.date() cache[key] = date b.append(date) def defaultdict(): cache = {} b= [] for i in ranges: b.append(cache.setdefault(i, i.date())) print "ifdict:", timeit.repeat("ifdict()", "from __main__ import ifdict", number=10000) print "keydict:", timeit.repeat("keydict()", "from __main__ import keydict", number=10000) print "defaultdict:", timeit.repeat("defaultdict()", "from __main__ import defaultdict", number=10000) </file> # python iforfile.py ifdict: [2.432887077331543, 2.4002890586853027, 2.397233009338379] keydict: [2.3483030796051025, 2.358638048171997, 2.314802885055542] defaultdict: [3.5384328365325928, 3.5329859256744385, 3.5728111267089844] # pypy iforfile.py (pypy 1.5) ifdict: [0.8129069805145264, 0.74648118019104, 0.7432689666748047] keydict: [0.5187451839447021, 0.4662129878997803, 0.4504108428955078] defaultdict: [37.98510789871216, 37.859113931655884, 37.92770600318909] Pypy displays significant slowdown in the defaultdict function, otherwise displays its usual speedup. To check what is the cause I replaced i.date() with i.day and found no major difference in times. It appears dict.setdefault (or it's interaction with jit) is causing a slow down. Regards

4 14

PyPy/NumPyPy sprint(s) in Leipzig, Germany?
by Mike Müller 20 Feb '12

20 Feb '12

Hi, I am just wondering if anybody is interested in sprinting on PyPy and in particular NumPyPy in Leipzig sometime in 2012. I can offer working space for up to 12 people with Wi-Fi as well as some basic catering (hot and cold drinks, snacks, pizza etc.) for a few days (up to a week). Accommodation in Leipzig is very reasonably priced. For example, there is a decent hotel very close to the venue for 35 Euros/night (single) or 50 Euros/night (double) including breakfast. Being a logistics center, Leipzig is easy to travel to by car, train or airplane including budget airlines. I would also act as co-sponsor (with the resources stated above) for an application for sprint funds (http://pythonsprints.com/cfa/ or other sources) that could cover (parts) of the traveling and accommodation expenses. Let me know what you think about it. I am open to ideas. Mike

5 31

Question about the STM module and RSTM
by Andrew Francis 01 Feb '12

01 Feb '12

Hi Folks: I understand that the STM module is still in its infancy. And subject to change at a moment's notice. I enjoy reading the transactional memory related blog posts. Over the weekend, I have been looking at the stm and transactionModule and the C++ based rstm. I haven't written an C++ based programme yet but that will change soon. If the stm library progresses a bit more, I would like to try writing Python versions of some of the STAMP examples. Until then, I am trying understand the relationship between stm and rstm. It does not seem that stm is simply a wrapper for rstm (a lot of work went into stm!). There are things in rstm that I see such as the ability to configure parameters, set policies and to get diagnostics (i.e., number of aborts). Are there plans to do this with the stm module? Cheers, Andrew

2 3

How will STM actually integrate with normal Python code
by Timothy Baldridge 31 Jan '12

31 Jan '12

As Armin stated in a recent mailing list thread: "In PyPy, we look at STM like we would look at the GC. It may be replaced in a week by a different one, but for the "end user" writing pure Python code, it essentially doesn't make a difference. " So, my question is, how exactly will STM integrate into PyPy? I'm going to take a guess here, and perhaps someone can elaborate to correct me. >From what I'm reading, PyPy with STM will offer the same promises (or lack of promises) that the JVM and CLR offer their code: For example, this code: def foo(d): if "foo" in d: del d["foo"] Will never cause a segmentation fault (due to multiple threads accessing "d" at the same time), but it may throw a KeyError. That is, all Python code will continue to be "thread-safe" as it is in CPython, but race conditions will continue to exist and must be handled in standard ways (locks, CAS, etc.). Am I right in this description? Thanks, Timothy -- “One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.” (Robert Firth)

2 1

Question about Atomic_ops.h in STM
by Andrew Francis 30 Jan '12

30 Jan '12

Hi Folks: I noticed a compare_and_swap function in atomic_ops.h. On the IRC channel, it was suggested that I look in _rffi_stm.py. However it is not there (I can understand thta). How can this function be exposed to a RPython programme? The reason I am asking is if compare and swap functions are available, it ought to be possible to write stuff like a lock-free list. Also this give me a chance to learn more about RPython. Cheers, Andrew

2 1

PyPy on Solaris?
by skip＠pobox.com 30 Jan '12

30 Jan '12

I'm trying to build PyPy (1.7) on Solaris and keep getting this error very early in the translate process: /opt/app/nonc++/gcc-4.5.2/lib/gcc/i386-pc-solaris2.10/4.5.2/include-fixed/sys/feature_tests.h:345:2: error: #error "Compiler or options invalid; UNIX 03 and POSIX.1-2001 applications require the use of c99" I saw a similar thread from last September on this topic which sort of suggested that there is no support for Solaris: http://comments.gmane.org/gmane.comp.python.pypy/8377 Has anything changed since then? If so, how do I craft my translate.py command line to get this to build? Thx, -- Skip Montanaro - skip(a)pobox.com - http://www.smontanaro.net/

2 1

JITability
by Serhat Sevki Dincer 30 Jan '12

30 Jan '12

Hi, Suppose you have a function to validate local/international phone numbers (one space between number groups, no space at start/end, no zero at start) like: def validatePhone(v): if not (9 < len(v) < 19 and '0' < v[0]): *# recall ' ' < '0'* return False for c in v: if '0' <= c <= '9': wasspace = False else: if c != ' ' or wasspace: return False wasspace = True if wasspace: return False # last one return True Just out of curiosity, would JITability of this function change if I put "wasspace = False" just before the loop ? Note: attaching just in case indentation is lost when sending..

4 4

SERGEY-MAVRODI.COM
by vwinfred＠mcdean.com 29 Jan '12

29 Jan '12

1 0

PyPy PPC64 Prolog translation
by David Edelsohn 26 Jan '12

26 Jan '12

Trying to keep everyone informed about what I learn while debugging the PPC backend and get some help with next step in analysis. The translated version of the Prolog interpreter (without JIT) runs for most examples, but crashes for on one. I recompiled the interpreter with "-O1 -g" to get a better traceback. $ pyrolog-c iter.pl >?- iterate_findall(10000). Program received signal SIGSEGV, Segmentation fault. pypy_g_throw (l_exc_1=0xfffb76a99c0, l_scont_4=<value optimized out>, l_fcont_4=<value optimized out>, l_heap_6=<value optimized out>) at interpreter_continuation.c:6580 6580 l_v161545 = (struct pypy_object_vtable0 *)_OP_GET_NEXT_GROUP_MEMBER((&pypy_g_typeinfo), (pypy_halfword_t)l_v161544->_gcheader.h_tid, sizeof(struct pypy_type_info0)); (gdb) where #0 pypy_g_throw (l_exc_1=0xfffb76a99c0, l_scont_4=<value optimized out>, l_fcont_4=<value optimized out>, l_heap_6=<value optimized out>) at interpreter_continuation.c:6580 #1 0x000000001010f8b8 in pypy_g_driver (l_scont_1=<value optimized out>, l_fcont_1=<value optimized out>, l_heap_2=<value optimized out>) at interpreter_continuation.c:3192 #2 0x0000000010116a30 in pypy_g_run (l_query_201=<value optimized out>, l_var_to_pos_4=0xfffb76a9748) at interpreter_translatedmain.c:2120 #3 0x000000001011861c in pypy_g_repl () at interpreter_translatedmain.c:231 #4 0x000000001010bf58 in pypy_g_entry_point (l_argv_1=<value optimized out>) at targetprologstandalone.c:242 #5 0x0000000010003708 in pypy_main_function (argc=<value optimized out>, argv=<value optimized out>) at /home/dje/src/pypy-ppc/pypy/translator/c/src/main.h:62 #6 0x00000fffb7c9f05c in .generic_start_main () from /lib64/power7/libc.so.6 #7 0x00000fffb7c9f27c in .__libc_start_main () from /lib64/power7/libc.so.6 #8 0x0000000000000000 in ?? () The following blocks from interpreter_continuation.c:pypy_g_throw() lead to the segfault, but I am not sure why the data structure is initialized with a bad value: 6574 block0: 6575 l_heap_689 = l_heap_6; 6576 l_scont_245 = l_scont_4; 6577 goto block1; 6578 6579 block1: 6580 l_v161066 = (struct pypy_object0 *)l_scont_245; 6581 l_v161067 = (struct pypy_object_vtable0 *)_OP_GET_NEXT_GROUP_MEMBER((&pypy_g_typeinfo), (pypy_halfword_t)l_v161066->_gcheader.h_tid, sizeof(struct pypy_type_info0)); 6582 l_v161068 = (struct pypy_prolog_interpreter_continuation_Continuation_vtabl0 *)l_v161067; 6583 l_v161069 = RPyField(l_v161068, c_cls_is_done); ... 6736 block19: 6737 l_v161116 = RPyField(l_scont_245, c_inst_nextcont); 6738 l_scont_245 = l_v161116; 6739 goto block1; l_scont_245 is correct upon initial entry (assigned from l_scont_4 argument). Execution flows through block19 3 times before the crash. On the third "iteration", l_scont_245 is set to 0, then dereferenced in block0, causing a segfault. pypy_g_throw() is entered with the chain of continuations (I assume that is what c_inst_nextcont means) already containing a NULL. >?- iterate_findall(10000). Breakpoint 1, pypy_g_throw (l_v161056=0xfffb76a99c0, l_scont_4=0xfffb76a98a0, l_fcont_4=0xfffb76a9748, l_heap_6=0xfffb76a9778) at interpreter_continuation.c:6581 (gdb) x/4x l_scont_4 0xfffb76a98a0: 0x00000000 0x00000fb0 0x00000fff 0xb76a9728 (gdb) x/4x 0xfffb76a9728 0xfffb76a9728: 0x00000000 0x000037f0 0x00000fff 0xb76a95e8 (gdb) x/4x 0xfffb76a95e8 0xfffb76a95e8: 0x00000000 0x00003ad8 0x00000000 0x00000000 The third and forth values are the 64 bit address of the next field in the chain. I am not sure how this chain of continuations is initialized, but one element contains a NULL pointer. Thanks, David

2 1

Re: [pypy-dev] [pypy-commit] pypy arm-backend-2: also remove test_compile_asmlen from runner_test after it was moved to the x86 backend
by Maciej Fijalkowski 26 Jan '12

26 Jan '12

Note that this is completely wrong - the test is precisely in runner_test because this is an expected JIT backend interface for jit hooks to work On Wed, Jan 18, 2012 at 1:35 PM, bivab <noreply(a)buildbot.pypy.org> wrote: > Author: David Schneider <david.schneider(a)picle.org> > Branch: arm-backend-2 > Changeset: r51441:839659291f03 > Date: 2012-01-18 12:33 +0100 > http://bitbucket.org/pypy/pypy/changeset/839659291f03/ > > Log: also remove test_compile_asmlen from runner_test after it was moved > to the x86 backend > > diff --git a/pypy/jit/backend/test/runner_test.py b/pypy/jit/backend/test/runner_test.py > --- a/pypy/jit/backend/test/runner_test.py > +++ b/pypy/jit/backend/test/runner_test.py > @@ -3188,55 +3188,6 @@ > res = self.cpu.get_latest_value_int(0) > assert res == -10 > > - def test_compile_asmlen(self): > - from pypy.jit.backend.llsupport.llmodel import AbstractLLCPU > - if not isinstance(self.cpu, AbstractLLCPU): > - py.test.skip("pointless test on non-asm") > - from pypy.jit.backend.x86.tool.viewcode import machine_code_dump > - import ctypes > - ops = """ > - [i2] > - i0 = same_as(i2) # but forced to be in a register > - label(i0, descr=1) > - i1 = int_add(i0, i0) > - guard_true(i1, descr=faildesr) [i1] > - jump(i1, descr=1) > - """ > - faildescr = BasicFailDescr(2) > - loop = parse(ops, self.cpu, namespace=locals()) > - faildescr = loop.operations[-2].getdescr() > - jumpdescr = loop.operations[-1].getdescr() > - bridge_ops = """ > - [i0] > - jump(i0, descr=jumpdescr) > - """ > - bridge = parse(bridge_ops, self.cpu, namespace=locals()) > - looptoken = JitCellToken() > - self.cpu.assembler.set_debug(False) > - info = self.cpu.compile_loop(loop.inputargs, loop.operations, looptoken) > - bridge_info = self.cpu.compile_bridge(faildescr, bridge.inputargs, > - bridge.operations, > - looptoken) > - self.cpu.assembler.set_debug(True) # always on untranslated > - assert info.asmlen != 0 > - cpuname = autodetect_main_model_and_size() > - # XXX we have to check the precise assembler, otherwise > - # we don't quite know if borders are correct > - > - def checkops(mc, ops): > - assert len(mc) == len(ops) > - for i in range(len(mc)): > - assert mc[i].split("\t")[-1].startswith(ops[i]) > - > - data = ctypes.string_at(info.asmaddr, info.asmlen) > - mc = list(machine_code_dump(data, info.asmaddr, cpuname)) > - lines = [line for line in mc if line.count('\t') == 2] > - checkops(lines, self.add_loop_instructions) > - data = ctypes.string_at(bridge_info.asmaddr, bridge_info.asmlen) > - mc = list(machine_code_dump(data, bridge_info.asmaddr, cpuname)) > - lines = [line for line in mc if line.count('\t') == 2] > - checkops(lines, self.bridge_loop_instructions) > - > > def test_compile_bridge_with_target(self): > # This test creates a loopy piece of code in a bridge, and builds another > _______________________________________________ > pypy-commit mailing list > pypy-commit(a)python.org > http://mail.python.org/mailman/listinfo/pypy-commit

2 1