[pypy-dev] pypyjit/test_pypy_c

[Manuel Jacob]

On 2014-01-21 11:34, Armin Rigo wrote:
> Hi all,
> 
> There are failures in pypyjit/test_pypy_c/test_string.py that shows
> that the JIT gets much worse code in some cases.  This is probably
> related to the refactor-str-types branch merge (at least the dates
> match).  I found and fixed one issue, which was that some code in
> stringmethods.py has a loop in the slow path, preventing the JIT from
> inlining the fast path.  There are still failures though.  I suppose I
> should mention again that merging big branches without checking all
> tests first is a Bad Thing to do.


Hi,

yes, it's right that the merging of refactor-str-types has broken three 
tests in pypy.module.pypyjit.test_pypy_c.test_string. I didn't know that 
there are tests in pypyjit/test_pypy_c that are not run by own-linux-*, 
so I didn't run pypy-c-jit-linux-*. Normally I check the buildbots on 
the next day to see if something is broken I didn't notice before. But 
test_decode_ascii() in the same test class is broken since over one 
month now. So let's come to the following point:

I'm in favor of merging branches only if all tests pass. What's 
preventing me from doing this is that the buildbots are broken in the 
general case. That makes it difficult to track which tests are broken by 
the branch or broken in default, too. As soon as one developers doesn't 
care about the buildbots and several tests fail, the whole continuous 
testing doesn't work (at least for me ;)). What can we do to prevent 
that tests are failing for more than one month?

As I mentioned on IRC a few days ago, the technical solution for this 
is to run the buildbots on every commit. Since the test suite takes a 
lot of time and we have limited resources, we can't do that. So we have 
to solve this in a non-technical way. Everyone should feel responsible 
for tests that fail and check whether he broke that test by accident. If 
not, he should feel free to blame others on IRC or the mailing list.

-Manuel