[pypy-dev] problem after merging of jit-virtual_state

[Hakan Ardo]

On Mon, Mar 14, 2011 at 11:49 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Hi Hakan,
> thank you for the deep explanation. Now I understand what's going on :-)
>
> So, I changed test_pypy_c_new to add a sys.setcheckinterval(some-huge-number),
> so that the bridge from the signal/thread counter is never created and we can
> forget about it.
>
> Now, if I understand correctly, the two remaining loops are one for the case
> "i non virtual" and the other for the case "i virtual", although both lead to
> the same operations. I think this is the expected behavior in this case, so
> are you ok if I just fix test_f1 to expect two loops?

Yes

>
> ciao,
> Anto
>
> On 13/03/11 11:12, Hakan Ardo wrote:
>> Hi,
>> this is what happens here:
>>
>> 1. The inner loop is traced and Loop0 is produced with preamble Loop1
>>
>> 2. A bridge from Guard3 (the test in the while) back to Loop0 is
>> traced (i.e the remaining parts of the outer loop)
>>
>> 3. At the end of this bridge the VirtualState does not match the
>> VirtualState of Loop0, so the loop is retraced
>>
>> 4. The VirtualState of the newly traced version of the loop does not
>> match the VirtualState at the end of the bridge so the bridge has to
>> jump to the preamble instead of jumping to the new specialized version
>> of the loop.
>>
>> 5. A bridge from Guard6 (signal/thread counter) is traced and the same
>> thing happens for this bridge.
>>
>> This means that the additional two versions of the loop will never be
>> used and should hopefully be reomved by the gc...
>>
>> So there are two issues:
>>
>> A. The additional specialized versions created does not become usable.
>> This is the issue I'm working on in the jit-usable_retrace branch. The
>> idea there is to have the retrace inherit the OptValue's of the
>> jumpargs at the end of the bridge. This will become a fairly large
>> change functionality wise...
>>
>> B. The VirtualStates' s differs in the first place forcing a retrace.
>> This is probably fixable by introducing some more cases in
>> NotVirtualInfo._generate_guards(). The jit-usable_retrace branch
>> contains more cases than trunk, don't know if those are enough for
>> this test though...
>>
>> Note however that
>> jit/metainterp/test/test_nested_loops_discovered_by_bridge in
>> test_loop_unroll.py, which conatins the same loop for a simple
>> interpreter, does work nicely, wihtout the issues above.
>>
>> On Sat, Mar 12, 2011 at 10:59 PM, Hakan Ardo <hakan at debian.org> wrote:
>>> On Sat, Mar 12, 2011 at 8:34 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>>>> Hi Hakan,
>>>>
>>>> On 12/03/11 19:25, Hakan Ardo wrote:
>>>>> Yes, this is probably the VirtualState checking. It will retrace a
>>>>> loop whenever the VirtualState at the end of a bridge differs from the
>>>>> VirtualState at the beginning of the compiled trace (any of the
>>>>> compiled traces). This might indeed produce an identical trace if we
>>>>> are unlucky, but the idea is that this should only happen rarely.
>>>>
>>>> ok, that's clear. So, hopefully this particular example looks a bit bad, but
>>>> in general it should not be an issue. It'd be nice to have a way to check this
>>>> thesis, but I agree that it's a bit hard.
>>>
>>> We should probably log the VirtualState together with the produced
>>> loops and bridges. That would allow us to see how they differ when a
>>> new version of a loop is traced. There are __repr__ methods I've been
>>> using for that while debugging. They might need some rpythonizing to
>>> translate though---
>>>
>>>>
>>>>> This is because the VirtualState  at the beginning of a trace is the
>>>>> state of all the OptValue of the inputargs produced during the
>>>>> optimization of the trace. This does not have to be the most general
>>>>> state for which the trace is usable (which would be hard to calculate
>>>>> I'm afraid).
>>>>
>>>> so, if I understand correctly, this is what happens:
>>>>
>>>> 1. we trace, optimize and compile loop A
>>>>
>>>> 2. after a while, we trace, optimize a compile a bridge B which then jumps
>>>> back to A; by chance, the bridge looks the same as the loop
>>>>
>>>> Am I right?
>>>
>>> Maybe, I've not had the chance to look into any details yet. I'll do
>>> that tomorrow...
>>>
>>>>
>>>>> A few cases that would (most likely) result in identical traces are
>>>>> salvaged in NotVirtualInfo._generate_guards by producing some extra
>>>>> gurads at the end of a bridge to make the VirtualState there match the
>>>>> VirtualState of a compiled trace. This is however only done if the
>>>>> guards would (most likely) not fail for the traced iteration.
>>>>>
>>>>> I'll look into what's happening in this particular test...
>>>>
>>>> I just did a quick check because I'm in a hurry, but from what I see we get
>>>> three actual *loops*, not bridges.
>>>
>>> So if it's the same loop traced several times they should all have the
>>> same preamble, and the preamble would have two bridges leading to the
>>> two second versions of the loop. The preamble and it's two bridges
>>> should end with different VirtualState's. The loops should  be
>>> specialized to the different VirtualState's, but if the VirtualState's
>>> are similar enough (but not equal) they might consist of the exact
>>> same operations.
>>>
>>> If there are 3 preambles for the same loop, there is a bug somewhere...
>>>
>>> --
>>> Håkan Ardö
>>>
>>
>>
>>
>
>



-- 
Håkan Ardö