[pypy-dev] problem after merging of jit-virtual_state

[Antonio Cuni]

Hi Hakan,
thank you for the deep explanation. Now I understand what's going on :-)

So, I changed test_pypy_c_new to add a sys.setcheckinterval(some-huge-number),
so that the bridge from the signal/thread counter is never created and we can
forget about it.

Now, if I understand correctly, the two remaining loops are one for the case
"i non virtual" and the other for the case "i virtual", although both lead to
the same operations. I think this is the expected behavior in this case, so
are you ok if I just fix test_f1 to expect two loops?

ciao,
Anto

On 13/03/11 11:12, Hakan Ardo wrote:
> Hi,
> this is what happens here:
> 
> 1. The inner loop is traced and Loop0 is produced with preamble Loop1
> 
> 2. A bridge from Guard3 (the test in the while) back to Loop0 is
> traced (i.e the remaining parts of the outer loop)
> 
> 3. At the end of this bridge the VirtualState does not match the
> VirtualState of Loop0, so the loop is retraced
> 
> 4. The VirtualState of the newly traced version of the loop does not
> match the VirtualState at the end of the bridge so the bridge has to
> jump to the preamble instead of jumping to the new specialized version
> of the loop.
> 
> 5. A bridge from Guard6 (signal/thread counter) is traced and the same
> thing happens for this bridge.
> 
> This means that the additional two versions of the loop will never be
> used and should hopefully be reomved by the gc...
> 
> So there are two issues:
> 
> A. The additional specialized versions created does not become usable.
> This is the issue I'm working on in the jit-usable_retrace branch. The
> idea there is to have the retrace inherit the OptValue's of the
> jumpargs at the end of the bridge. This will become a fairly large
> change functionality wise...
> 
> B. The VirtualStates' s differs in the first place forcing a retrace.
> This is probably fixable by introducing some more cases in
> NotVirtualInfo._generate_guards(). The jit-usable_retrace branch
> contains more cases than trunk, don't know if those are enough for
> this test though...
> 
> Note however that
> jit/metainterp/test/test_nested_loops_discovered_by_bridge in
> test_loop_unroll.py, which conatins the same loop for a simple
> interpreter, does work nicely, wihtout the issues above.
> 
> On Sat, Mar 12, 2011 at 10:59 PM, Hakan Ardo <hakan at debian.org> wrote:
>> On Sat, Mar 12, 2011 at 8:34 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
>>> Hi Hakan,
>>>
>>> On 12/03/11 19:25, Hakan Ardo wrote:
>>>> Yes, this is probably the VirtualState checking. It will retrace a
>>>> loop whenever the VirtualState at the end of a bridge differs from the
>>>> VirtualState at the beginning of the compiled trace (any of the
>>>> compiled traces). This might indeed produce an identical trace if we
>>>> are unlucky, but the idea is that this should only happen rarely.
>>>
>>> ok, that's clear. So, hopefully this particular example looks a bit bad, but
>>> in general it should not be an issue. It'd be nice to have a way to check this
>>> thesis, but I agree that it's a bit hard.
>>
>> We should probably log the VirtualState together with the produced
>> loops and bridges. That would allow us to see how they differ when a
>> new version of a loop is traced. There are __repr__ methods I've been
>> using for that while debugging. They might need some rpythonizing to
>> translate though---
>>
>>>
>>>> This is because the VirtualState  at the beginning of a trace is the
>>>> state of all the OptValue of the inputargs produced during the
>>>> optimization of the trace. This does not have to be the most general
>>>> state for which the trace is usable (which would be hard to calculate
>>>> I'm afraid).
>>>
>>> so, if I understand correctly, this is what happens:
>>>
>>> 1. we trace, optimize and compile loop A
>>>
>>> 2. after a while, we trace, optimize a compile a bridge B which then jumps
>>> back to A; by chance, the bridge looks the same as the loop
>>>
>>> Am I right?
>>
>> Maybe, I've not had the chance to look into any details yet. I'll do
>> that tomorrow...
>>
>>>
>>>> A few cases that would (most likely) result in identical traces are
>>>> salvaged in NotVirtualInfo._generate_guards by producing some extra
>>>> gurads at the end of a bridge to make the VirtualState there match the
>>>> VirtualState of a compiled trace. This is however only done if the
>>>> guards would (most likely) not fail for the traced iteration.
>>>>
>>>> I'll look into what's happening in this particular test...
>>>
>>> I just did a quick check because I'm in a hurry, but from what I see we get
>>> three actual *loops*, not bridges.
>>
>> So if it's the same loop traced several times they should all have the
>> same preamble, and the preamble would have two bridges leading to the
>> two second versions of the loop. The preamble and it's two bridges
>> should end with different VirtualState's. The loops should  be
>> specialized to the different VirtualState's, but if the VirtualState's
>> are similar enough (but not equal) they might consist of the exact
>> same operations.
>>
>> If there are 3 preambles for the same loop, there is a bug somewhere...
>>
>> --
>> Håkan Ardö
>>
> 
> 
>