[pypy-svn] r70534 - in pypy/trunk/pypy: interpreter jit/backend jit/metainterp jit/metainterp/test module/pypyjit/test rlib

cfbolz at codespeak.net cfbolz at codespeak.net
Tue Jan 12 16:49:26 CET 2010


Author: cfbolz
Date: Tue Jan 12 16:49:26 2010
New Revision: 70534

Modified:
   pypy/trunk/pypy/interpreter/baseobjspace.py
   pypy/trunk/pypy/jit/backend/model.py
   pypy/trunk/pypy/jit/metainterp/codewriter.py
   pypy/trunk/pypy/jit/metainterp/optimizeopt.py
   pypy/trunk/pypy/jit/metainterp/pyjitpl.py
   pypy/trunk/pypy/jit/metainterp/resoperation.py
   pypy/trunk/pypy/jit/metainterp/test/test_basic.py
   pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
   pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
   pypy/trunk/pypy/rlib/jit.py
Log:
merge loop-invariant-decorator branch:

    ------------------------------------------------------------------------            
    r70448 | cfbolz | 2010-01-08 12:22:09 +0100 (Fri, 08 Jan 2010) | 3 lines            
    Changed paths:                                                                      
       A /pypy/branch/loop-invariant-decorator (from /pypy/trunk:70447)                 

    A new branch to implement the loop_invariant decorator needed to deal with
    reading the executioncontext out of TLS.                                  

    ------------------------------------------------------------------------
    r70499 | cfbolz | 2010-01-11 14:59:10 +0100 (Mon, 11 Jan 2010) | 2 lines
    Changed paths:                                                          
       M /pypy/branch/loop-invariant-decorator/pypy/rlib/jit.py             

    add the loop_invariant decorator

    ------------------------------------------------------------------------
    r70500 | cfbolz | 2010-01-11 15:00:11 +0100 (Mon, 11 Jan 2010) | 2 lines
    Changed paths:                                                          
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/test/test_basic.py

    the test that I would like to pass

    ------------------------------------------------------------------------
    r70501 | cfbolz | 2010-01-11 15:13:21 +0100 (Mon, 11 Jan 2010) | 4 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/jit/backend/model.py
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/resoperation.py

    Add a new operation CALL_LOOPINVARIANT. The operation behaves exactly like CALL.
    The idea is that the backend never has to generate code for this operation,
    because the optimizer turns it into a normal CALL.

    ------------------------------------------------------------------------
    r70502 | cfbolz | 2010-01-11 15:15:16 +0100 (Mon, 11 Jan 2010) | 3 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/codewriter.py
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/pyjitpl.py

    Make the codewriter produce a new opcode, "residual_call_loopinvariant" and
    implement it in the metainterp.

    ------------------------------------------------------------------------
    r70503 | cfbolz | 2010-01-11 15:18:53 +0100 (Mon, 11 Jan 2010) | 4 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/optimizeopt.py
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/test/test_optimizeopt.py

    An optimization that replaces several loop-invariant calls to the same function
    by one call. Also add a direct test for that optimization. This implements the
    original test.

    ------------------------------------------------------------------------
    r70504 | cfbolz | 2010-01-11 15:20:44 +0100 (Mon, 11 Jan 2010) | 2 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/interpreter/baseobjspace.py

    Declare getexecutioncontext to be loop-invariant.

    ------------------------------------------------------------------------
    r70505 | cfbolz | 2010-01-11 15:27:39 +0100 (Mon, 11 Jan 2010) | 2 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/interpreter/baseobjspace.py

    missing import, of course

    ------------------------------------------------------------------------
    r70508 | cfbolz | 2010-01-11 16:10:50 +0100 (Mon, 11 Jan 2010) | 2 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/optimizeopt.py
       M /pypy/branch/loop-invariant-decorator/pypy/jit/metainterp/test/test_optimizeopt.py

    Called functions are ConstAddrs, use the address itself as keys.

    ------------------------------------------------------------------------
    r70530 | cfbolz | 2010-01-12 15:00:29 +0100 (Tue, 12 Jan 2010) | 2 lines
    Changed paths:
       M /pypy/branch/loop-invariant-decorator/pypy/module/pypyjit/test/test_pypy_c.py

    Adapt the test to be fine with the threading stuff.

    ------------------------------------------------------------------------



Modified: pypy/trunk/pypy/interpreter/baseobjspace.py
==============================================================================
--- pypy/trunk/pypy/interpreter/baseobjspace.py	(original)
+++ pypy/trunk/pypy/interpreter/baseobjspace.py	Tue Jan 12 16:49:26 2010
@@ -9,7 +9,7 @@
 from pypy.rlib.objectmodel import we_are_translated
 from pypy.rlib.debug import make_sure_not_resized
 from pypy.rlib.timer import DummyTimer, Timer
-from pypy.rlib.jit import unroll_safe
+from pypy.rlib import jit
 import os, sys
 
 __all__ = ['ObjSpace', 'OperationError', 'Wrappable', 'W_Root']
@@ -531,6 +531,7 @@
     def leave_cache_building_mode(self, val):
         "hook for the flow object space"
 
+    @jit.loop_invariant
     def getexecutioncontext(self):
         "Return what we consider to be the active execution context."
         # Important: the annotator must not see a prebuilt ExecutionContext:
@@ -735,7 +736,7 @@
         """
         return self.unpackiterable(w_iterable, expected_length)
 
-    @unroll_safe
+    @jit.unroll_safe
     def exception_match(self, w_exc_type, w_check_class):
         """Checks if the given exception type matches 'w_check_class'."""
         if self.is_w(w_exc_type, w_check_class):

Modified: pypy/trunk/pypy/jit/backend/model.py
==============================================================================
--- pypy/trunk/pypy/jit/backend/model.py	(original)
+++ pypy/trunk/pypy/jit/backend/model.py	Tue Jan 12 16:49:26 2010
@@ -209,6 +209,9 @@
     def do_call(self, args, calldescr):
         raise NotImplementedError
 
+    def do_call_loopinvariant(self, args, calldescr):
+        return self.do_call(args, calldescr)
+
     def do_cond_call_gc_wb(self, args, calldescr):
         if args[0].getint() & args[1].getint():
             self.do_call(args[2:], calldescr)

Modified: pypy/trunk/pypy/jit/metainterp/codewriter.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/codewriter.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/codewriter.py	Tue Jan 12 16:49:26 2010
@@ -1235,9 +1235,11 @@
         calldescr, non_void_args = self.codewriter.getcalldescr(
             op.args[0], args, op.result, consider_effects_of=op)
         pure = False
+        loopinvariant = False
         if op.opname == "direct_call":
             func = getattr(get_funcobj(op.args[0].value), '_callable', None)
             pure = getattr(func, "_pure_function_", False)
+            loopinvariant = getattr(func, "_jit_loop_invariant_", False)
             all_promoted_args = getattr(func,
                                "_pure_function_with_all_promoted_args_", False)
             if pure and not all_promoted_args:
@@ -1249,7 +1251,10 @@
         except lltype.DelayedPointer:
             canraise = True  # if we need to look into the delayed ptr that is
                              # the portal, then it's certainly going to raise
-        if pure:
+        if loopinvariant:
+            self.emit("residual_call_loopinvariant")
+            assert not non_void_args, "arguments not supported for loop-invariant function!"
+        elif pure:
             # XXX check what to do about exceptions (also MemoryError?)
             self.emit('residual_call_pure')
         elif canraise:

Modified: pypy/trunk/pypy/jit/metainterp/optimizeopt.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/optimizeopt.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/optimizeopt.py	Tue Jan 12 16:49:26 2010
@@ -388,6 +388,7 @@
         self.resumedata_memo = resume.ResumeDataLoopMemo(metainterp_sd)
         self.heap_op_optimizer = HeapOpOptimizer(self)
         self.bool_boxes = {}
+        self.loop_invariant_results = {}
 
     def forget_numberings(self, virtualbox):
         self.metainterp_sd.profiler.count(jitprof.OPT_FORCINGS)
@@ -874,6 +875,23 @@
     def optimize_DEBUG_MERGE_POINT(self, op):
         self.emit_operation(op)
 
+    def optimize_CALL_LOOPINVARIANT(self, op):
+        funcvalue = self.getvalue(op.args[0])
+        if not funcvalue.is_constant():
+            self.optimize_default(op)
+            return
+        resvalue = self.loop_invariant_results.get(op.args[0].getint(), None)
+        if resvalue is not None:
+            self.make_equal_to(op.result, resvalue)
+            return
+        # change the op to be a normal call, from the backend's point of view
+        # there is no reason to have a separate operation for this
+        op.opnum = rop.CALL
+        self.optimize_default(op)
+        resvalue = self.getvalue(op.result)
+        self.loop_invariant_results[op.args[0].getint()] = resvalue
+            
+
 optimize_ops = _findall(Optimizer, 'optimize_')
 
 

Modified: pypy/trunk/pypy/jit/metainterp/pyjitpl.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/pyjitpl.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/pyjitpl.py	Tue Jan 12 16:49:26 2010
@@ -654,6 +654,10 @@
     def opimpl_residual_call(self, calldescr, varargs):
         return self.do_residual_call(varargs, descr=calldescr, exc=True)
 
+    @arguments("descr", "varargs")
+    def opimpl_residual_call_loopinvariant(self, calldescr, varargs):
+        return self.execute_varargs(rop.CALL_LOOPINVARIANT, varargs, calldescr, exc=True)
+
     @arguments("varargs")
     def opimpl_recursion_leave_prep(self, varargs):
         warmrunnerstate = self.metainterp.staticdata.state

Modified: pypy/trunk/pypy/jit/metainterp/resoperation.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/resoperation.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/resoperation.py	Tue Jan 12 16:49:26 2010
@@ -226,6 +226,7 @@
     '_CANRAISE_FIRST', # ----- start of can_raise operations -----
     'CALL',
     'CALL_MAY_FORCE',
+    'CALL_LOOPINVARIANT',
     'OOSEND',                     # ootype operation
     '_CANRAISE_LAST', # ----- end of can_raise operations -----
 

Modified: pypy/trunk/pypy/jit/metainterp/test/test_basic.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_basic.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_basic.py	Tue Jan 12 16:49:26 2010
@@ -1,7 +1,7 @@
 import py
 import sys
 from pypy.rlib.jit import JitDriver, we_are_jitted, hint, dont_look_inside
-from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE
+from pypy.rlib.jit import OPTIMIZER_FULL, OPTIMIZER_SIMPLE, loop_invariant
 from pypy.jit.metainterp.warmspot import ll_meta_interp, get_stats
 from pypy.jit.backend.llgraph import runner
 from pypy.jit.metainterp import support, codewriter, pyjitpl, history
@@ -1221,6 +1221,34 @@
         res = self.meta_interp(f, [21])
         assert res == 42
         self.check_loops(guard_nonnull=1, guard_isnull=1)
+
+    def test_loop_invariant(self):
+        myjitdriver = JitDriver(greens = [], reds = ['x', 'res'])
+        class A(object):
+            pass
+        a = A()
+        a.current_a = A()
+        a.current_a.x = 1
+        @loop_invariant
+        def f():
+            return a.current_a
+
+        def g(x):
+            res = 0
+            while x > 0:
+                myjitdriver.can_enter_jit(x=x, res=res)
+                myjitdriver.jit_merge_point(x=x, res=res)
+                res += f().x
+                res += f().x
+                res += f().x
+                x -= 1
+            a.current_a = A()
+            a.current_a.x = 2
+            return res
+        res = self.meta_interp(g, [21])
+        assert res == 3 * 21
+        self.check_loops(call=1)
+
         
 
 class TestOOtype(BasicTests, OOJitMixin):

Modified: pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py
==============================================================================
--- pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py	(original)
+++ pypy/trunk/pypy/jit/metainterp/test/test_optimizeopt.py	Tue Jan 12 16:49:26 2010
@@ -648,6 +648,32 @@
 
     # ----------
 
+    def test_call_loopinvariant(self):
+        ops = """
+        [i1]
+        i2 = call_loopinvariant(1, i1, descr=nonwritedescr)
+        guard_no_exception() []
+        guard_value(i2, 1) []
+        i3 = call_loopinvariant(1, i1, descr=nonwritedescr)
+        guard_no_exception() []
+        guard_value(i2, 1) []
+        i4 = call_loopinvariant(1, i1, descr=nonwritedescr)
+        guard_no_exception() []
+        guard_value(i2, 1) []
+        jump(i1)
+        """
+        expected = """
+        [i1]
+        i2 = call(1, i1, descr=nonwritedescr)
+        guard_no_exception() []
+        guard_value(i2, 1) []
+        jump(i1)
+        """
+        self.optimize_loop(ops, 'Not', expected)
+
+
+    # ----------
+
     def test_virtual_1(self):
         ops = """
         [i, p0]

Modified: pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py
==============================================================================
--- pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py	(original)
+++ pypy/trunk/pypy/module/pypyjit/test/test_pypy_c.py	Tue Jan 12 16:49:26 2010
@@ -172,7 +172,7 @@
                         x = x + (i&j)
                     i = i + 1
                 return x
-        ''', 194,
+        ''', 220,
                    ([2117], 1083876708))
 
     def test_factorial(self):
@@ -183,7 +183,7 @@
                     r *= n
                     n -= 1
                 return r
-        ''', 26,
+        ''', 28,
                    ([5], 120),
                     ([20], 2432902008176640000L))
 
@@ -237,8 +237,11 @@
         assert not ops[4]
         ops = self.get_by_bytecode("CALL_FUNCTION")
         assert len(ops) == 2
-        for bytecode in ops:
-            assert not bytecode.get_opnames("call")
+        for i, bytecode in enumerate(ops):
+            if i == 0:
+                assert "call(getexecutioncontext)" in str(bytecode)
+            else:
+                assert not bytecode.get_opnames("call")
             assert not bytecode.get_opnames("new")
             assert len(bytecode.get_opnames("guard")) <= 10
 
@@ -268,8 +271,11 @@
 
         ops = self.get_by_bytecode("CALL_METHOD")
         assert len(ops) == 2
-        for bytecode in ops:
-            assert not bytecode.get_opnames("call")
+        for i, bytecode in enumerate(ops):
+            if i == 0:
+                assert "call(getexecutioncontext)" in str(bytecode)
+            else:
+                assert not bytecode.get_opnames("call")
             assert not bytecode.get_opnames("new")
             assert len(bytecode.get_opnames("guard")) <= 9
         assert len(ops[1]) < len(ops[0])
@@ -296,8 +302,11 @@
                    ([31], 32))
         ops = self.get_by_bytecode("CALL_FUNCTION")
         assert len(ops) == 2
-        for bytecode in ops:
-            assert not bytecode.get_opnames("call")
+        for i, bytecode in enumerate(ops):
+            if i == 0:
+                assert "call(getexecutioncontext)" in str(bytecode)
+            else:
+                assert not bytecode.get_opnames("call")
             assert not bytecode.get_opnames("new")
         assert len(ops[0].get_opnames("guard")) <= 14
         assert len(ops[1].get_opnames("guard")) <= 3
@@ -315,7 +324,7 @@
                     a.x = 2
                     i = i + a.x
                 return i
-        ''', 65,
+        ''', 67,
                    ([20], 20),
                    ([31], 32))
 
@@ -346,7 +355,7 @@
                 while i < n:
                     i = i + a.x
                 return i
-        ''', 39,
+        ''', 41,
                    ([20], 20),
                    ([31], 32))
 
@@ -459,8 +468,6 @@
             py.test.skip("pass --pypy!")
         if not has_info(option.pypy_c, 'translation.jit'):
             py.test.skip("must give a pypy-c with the jit enabled")
-        if has_info(option.pypy_c, 'translation.thread'):
-            py.test.skip("for now, must give a pypy-c-jit without threads")
         cls.tmpdir = udir.join('pypy-jit')
         cls.tmpdir.ensure(dir=1)
         cls.counter = 0

Modified: pypy/trunk/pypy/rlib/jit.py
==============================================================================
--- pypy/trunk/pypy/rlib/jit.py	(original)
+++ pypy/trunk/pypy/rlib/jit.py	Tue Jan 12 16:49:26 2010
@@ -20,6 +20,12 @@
     func._jit_unroll_safe_ = True
     return func
 
+def loop_invariant(func):
+    dont_look_inside(func)
+    func._jit_loop_invariant_ = True
+    return func
+
+
 def purefunction_promote(func):
     import inspect
     purefunction(func)



More information about the Pypy-commit mailing list