[pypy-dev] Spurious dict lookups in my JIT loops
Boris
boris2010 at boristhebrave.com
Sat Sep 17 23:38:43 CEST 2011
It's a little long for that, I was hoping people would request what is
relevant. Here's some pertinent snippets (again, this is all an experiment,
so it is not pretty).
jitdriver = JitDriver(greens=['pc', 'method'],
reds=['self'],
virtualizables=['self'],
get_printable_location=get_location)
class Frame:
""" Represents one stack frame."""
_immutable_fields_ = ['space', 'method', 'opStack', 'scopeStack',
'locals']
_virtualizable2_ = ["locals[*]"]
def __init__(self, space, method):
self = jit.hint(self, access_directly=True,
fresh_virtualizable=True)
self.space = space
self.method = method
body = method.method_body
self.opStack = [space.getUndefined()] * body.max_stack
self.opCount = 0
self.scopeStack = [space.getUndefined()] * body.max_scope_depth
self.scopeCount = 0
self.locals = [space.getUndefined()] * body.local_count
def dispatch(self):
self = jit.hint(self, access_directly=True)
pc = 0
while True:
jitdriver.jit_merge_point(self=self, pc=pc, method=self.method)
self.opCount = jit.hint(self.opCount, promote=True)
self.scopeCount = jit.hint(self.scopeCount, promote=True)
r, pc = self.handle_bytecode(pc)
if r is not None:
return r
def handle_bytecode(self, pc):
"""Runs the interpreter for a single bytecode.
Returns (retvalue,pc) where retvalue is non-None if if the RETURNVALUE or
RETURNVOID opcodes are run, and pc the new program counter."""
bytecode = ord(self.method.method_body.code[pc])
pc+=1
if bytecode == Frame.LABEL:
pass
elif bytecode == Frame.INCLOCAL_I:
index, pc = readU30(self.method.method_body.code, pc)
self.locals[force_non_neg(index)] =
self.space.wrapInt(self.space.toInteger(self.locals[force_non_neg(index)]) +
1)
elif bytecode == Frame.GETLOCAL2:
self.push(self.locals[2])
elif bytecode == Frame.PUSHSHORT:
i, pc = readU30(self.method.method_body.code, pc)
self.push(self.space.wrapInt(i))
elif (bytecode == Frame.IFGE or
bytecode == Frame.IFGT or
bytecode == Frame.IFLE or
bytecode == Frame.IFLT or
bytecode == Frame.IFNGE or
bytecode == Frame.IFNGT or
bytecode == Frame.IFNLE or
bytecode == Frame.IFNLT):
offset, pc = readSI24(self.method.method_body.code, pc)
b = self.pop()
a = self.pop()
c = self.compare(a, b)
doBranch = False
if c == -99:
doBranch = False
else:
if bytecode in (Frame.IFGE, Frame.IFNGE): doBranch = c >=
0
elif bytecode in (Frame.IFGT, Frame.IFNGT): doBranch = c >
0
elif bytecode in (Frame.IFLE, Frame.IFNLE): doBranch = c <=
0
elif bytecode in (Frame.IFLT, Frame.IFNLT): doBranch = c <
0
if bytecode in
(Frame.IFNGE,Frame.IFNGT,Frame.IFNLE,Frame.IFNLT):
doBranch = not doBranch
if doBranch:
pc += offset
#jitdriver.can_enter_jit(self=self, pc=pc,
method=self.method)
The bytecode in question:
L1: label
inclocal_i 2
L0: getlocal2
pushshort 10000
iflt L1
(i.e. increment local variable 2 by 1, then compare it against 10000)
On Sat, Sep 17, 2011 at 6:25 PM, Benjamin Peterson <benjamin at python.org>wrote:
> This would probably be easier if you showed us the code.
>
> 2011/9/17 Boris <boris2010 at boristhebrave.com>:
> > Hi,
> >
> > I've been trying out writing my own interpreter using the PyPy framework
> > recently, as a bit of fun. I've been trying to get the JIT to optimize a
> > trivial loop down to the minimal amount of operations. With judicious use
> of
> > `_immutable_fields_` and `_virtualizable2_`, I've got pretty close.
> >
> > But I'm still seeing lots of calls to
> > `ll_dict_lookup__dicttablePtr_Signed_Signed`, which don't correspond to
> any
> > code in my interpreter. I don't think I even have any dicts that take
> > integer keys. Could someone give me a hint where these are coming from
> and
> > for what purpose? Or perhaps how to inspect the dicts or get further
> info?
> >
> > I append what I'm seeing in the logs (Note: I've augmented the logs to
> give
> > the raw pointer). In this case, it is only looking up the value 21, but
> I've
> > seen other values in addition when running other programs. The
> > setarrayitem_gc calls are expected - it is nulling out the stack that was
> > being used. Everything from i17 is unexpected. I tested on revisions
> > 00711ff1e03d and 96a212b0688a.
> >
> > Thanks,
> >
> > Boris
> >
> >
> > #############################
> >
> > [3fd12ea6569db] {jit-log-opt-loop
> > # Loop 0 : loop with 37 ops
> > [p0, p1, p2, i3, p4, p5]
> > debug_merge_point(0, '::Test$iinit:20')
> > +113: i7 = int_add(i3, 1)
> > debug_merge_point(0, '::Test$iinit:22')
> > debug_merge_point(0, '::Test$iinit:23')
> > debug_merge_point(0, '::Test$iinit:26')
> > +116: setarrayitem_gc(p5, 0, ConstPtr(ptr9,0x0), descr=<GcPtrArrayDescr>)
> > +126: setarrayitem_gc(p5, 1, ConstPtr(ptr11,0x0),
> descr=<GcPtrArrayDescr>)
> > +133: i13 = uint_lt(i7, 10000)
> > guard_true(i13, descr=<Guard2>) [p0, p1, p2, p4, i7]
> > +145: i17 = call(ConstClass(ll_dict_lookup__dicttablePtr_Signed_Signed),
> > ConstPtr(ptr15,0x84c2958), 21, 21, descr=<SignedCallDescr>)
> > +176: guard_no_exception(, descr=<Guard3>) [p0, i17, p1, p2, p4, i7]
> > +189: i19 = int_and(i17, -2147483648)
> > +195: i20 = int_is_true(i19)
> > guard_true(i20, descr=<Guard4>) [p0, p1, p2, p4, i7]
> > +204: i23 = call(ConstClass(ll_dict_lookup__dicttablePtr_Signed_Signed),
> > ConstPtr(ptr22,0x84c2978), 21, 21, descr=<SignedCallDescr>)
> > +235: guard_no_exception(, descr=<Guard5>) [p0, i23, p1, p2, p4, i7]
> > +248: i24 = int_and(i23, -2147483648)
> > +254: i25 = int_is_true(i24)
> > guard_true(i25, descr=<Guard6>) [p0, p1, p2, p4, i7]
> > +263: i28 = call(ConstClass(ll_dict_lookup__dicttablePtr_Signed_Signed),
> > ConstPtr(ptr27,0x84c2988), 21, 21, descr=<SignedCallDescr>)
> > +294: guard_no_exception(, descr=<Guard7>) [p0, i28, p1, p2, p4, i7]
> > +307: i29 = int_and(i28, -2147483648)
> > +313: i30 = int_is_true(i29)
> > guard_true(i30, descr=<Guard8>) [p0, p1, p2, p4, i7]
> > +322: i33 = call(ConstClass(ll_dict_lookup__dicttablePtr_Signed_Signed),
> > ConstPtr(ptr32,0x84c2998), 21, 21, descr=<SignedCallDescr>)
> > +353: guard_no_exception(, descr=<Guard9>) [p0, i33, p1, p2, p4, i7]
> > +366: i34 = int_and(i33, -2147483648)
> > +372: i35 = int_is_true(i34)
> > guard_false(i35, descr=<Guard10>) [p0, p1, p2, p4, i7]
> > +381: i38 = call(ConstClass(ll_dict_lookup__dicttablePtr_Signed_Signed),
> > ConstPtr(ptr37,0x84c2968), 21, 21, descr=<SignedCallDescr>)
> > +412: guard_no_exception(, descr=<Guard11>) [p0, i38, p1, p2, p4, i7]
> > +425: i39 = int_and(i38, -2147483648)
> > +431: i40 = int_is_true(i39)
> > guard_true(i40, descr=<Guard12>) [p0, p1, p2, p4, i7]
> > debug_merge_point(0, '::Test$iinit:20')
> > +440: i41 = arraylen_gc(p5, descr=<GcPtrArrayDescr>)
> > +440: jump(p0, p1, p2, i7, p4, p5, descr=<Loop0>)
> > +448: --end of the loop--
> > [3fd12ea696ce9] jit-log-opt-loop}
> >
> >
> >
> > _______________________________________________
> > pypy-dev mailing list
> > pypy-dev at python.org
> > http://mail.python.org/mailman/listinfo/pypy-dev
> >
> >
>
>
>
> --
> Regards,
> Benjamin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20110917/2b325471/attachment-0001.html>
More information about the pypy-dev
mailing list