Hello PyPy folks.
I have recently discovered that in Fedora, when building PyPy, we run
test.regrtest but we don't make test failures terminate the build.
For a couple times, I've reported test hangs because that was unfortunately the
only problem that we were able to spot.
Tests that run but don't fail the build are kinda useless. I want to remedy
that but I noticed that many tests fail for quite obvious reasons, for example:
======================================================================
FAIL: test_getsitepackages (test.test_site.HelperFunctionsTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/builddir/build/BUILD/pypy3.10-v7.3.19-src/lib-python/3/test/test_site.py",
line 335, in test_getsitepackages
self.assertEqual(dirs[0], wanted)
AssertionError: 'xoxo/lib64/pypy3.10/site-packages' !=
'xoxo/lib64/python3.10/site-packages'
- xoxo/lib64/pypy3.10/site-packages
? ^^
+ xoxo/lib64/python3.10/site-packages
? ^^^^
----------------------------------------------------------------------
I have examined the GitHub actions configuration in pypy/pypy but I don't see
the tests run.
What are the expectations about the test results? Should we check that the
failure rate does not exceed some percentage? Or should we skip know failures
one by one? Or report them to you?
Or do we keep running the tests just that we could see the failure in the logs
but don't fail the build for that?
How do you use those tests in PyPy?
Thanks,
--
Miro Hrončok
--
Phone: +420777974800
Fedora Matrix: mhroncok
