Hi, We have a "PPC64 AIX 3.x" buildbot slave which fails on cloning the GitHub repository: "SSL certificate problem: unable to get local issuer certificate". It started to fail around Feb 11, 2017 (Build #294), probably when buildbots moved to GitHub, after CPython moved to GitHub. First build which failed: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/294 Moreover, some tests are failing since at least 2 years on AIX. Some examples: * test_locale.test_strcoll_with_diacritic() * test_socket.testIPv4toString() * test_strptime Last build which ran unit tests: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/293/steps/t... For me, the principle of a CI is to detect *regressions*. But the AIX buildbot is always failing because of known bugs. There are 3 options: * Find a maintainer who quickly fix all known bugs. Unlike * Skip tests known to fail on AIX. I know that many core developers dislike this option, "hiding" bugs. * Drop the buildbot My favorite option is to drop the buildbot, since I'm tired of these red buildbot slaves. I added David Edelsohn to this email, he owns the buildbot, and I know that he wants to get the best AIX support in Python. The question is also which kind of support level do we provide per platform? Full support, "best effort" or no support? * Full support requires active maintainers, a CI with tests which pass, etc. * "Best effort": fix bugs when someone complains and someone (else?) provides a fix * No support: reject proposed patches to add a partial support for a platform. Victor
I would prefer that AIX not be dropped from support and from the buildbots.
I have fixed the Git problem. I thought that it had been working.
The testsuite failures on AIX are issues with the AIX kernel and C
Library, often corner cases. I don't want to get into arguments about
the POSIX standard. Some of the issues are actual conformance issues
and some are different interpretations of the standard. Addressing
the problems in AIX is a slow process. If the failing testcases are
too annoying, I would recommend to skip the testcases.
Despite the testsuite failures, Python builds and runs on AIX for the
vast majority of users and applications. I don't see the benefit in
dropping support for a platform that functions because it doesn't
fully pass the testsuite.
Thanks, David
On Mon, Apr 3, 2017 at 6:56 AM, Victor Stinner
Hi,
We have a "PPC64 AIX 3.x" buildbot slave which fails on cloning the GitHub repository: "SSL certificate problem: unable to get local issuer certificate". It started to fail around Feb 11, 2017 (Build #294), probably when buildbots moved to GitHub, after CPython moved to GitHub.
First build which failed: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/294
Moreover, some tests are failing since at least 2 years on AIX. Some examples:
* test_locale.test_strcoll_with_diacritic() * test_socket.testIPv4toString() * test_strptime
Last build which ran unit tests: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/293/steps/t...
For me, the principle of a CI is to detect *regressions*. But the AIX buildbot is always failing because of known bugs. There are 3 options:
* Find a maintainer who quickly fix all known bugs. Unlike * Skip tests known to fail on AIX. I know that many core developers dislike this option, "hiding" bugs. * Drop the buildbot
My favorite option is to drop the buildbot, since I'm tired of these red buildbot slaves.
I added David Edelsohn to this email, he owns the buildbot, and I know that he wants to get the best AIX support in Python.
The question is also which kind of support level do we provide per platform? Full support, "best effort" or no support?
* Full support requires active maintainers, a CI with tests which pass, etc. * "Best effort": fix bugs when someone complains and someone (else?) provides a fix * No support: reject proposed patches to add a partial support for a platform.
Victor
2017-04-03 16:00 GMT+02:00 David Edelsohn
I have fixed the Git problem. I thought that it had been working.
Thank you :-) I see tests running: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/567/steps/t...
The testsuite failures on AIX are issues with the AIX kernel and C Library, often corner cases. I don't want to get into arguments about the POSIX standard. Some of the issues are actual conformance issues and some are different interpretations of the standard.
Many tests of the CPython test suite actually test also the OS, not only Python. But in many cases, it's complex to "mock" the OS (unittest.mock), since tested functions are thin wrapper to syscals.
Addressing the problems in AIX is a slow process. If the failing testcases are too annoying, I would recommend to skip the testcases.
So yeah, let's skip such tests on AIX!
Despite the testsuite failures, Python builds and runs on AIX for the vast majority of users and applications. I don't see the benefit in dropping support for a platform that functions because it doesn't fully pass the testsuite.
It's a limitation of buildbot, the output is basically binary: pass (green) or fail (red). If buildbot would understand test results (parse stdout, or Python produces a output file easier to parse), it would be possible to detect *regressions* compared to previous runs. Using Jenkins and JUnit, I know that it's possible to do that. But I'm not volunteer to handle buildbot or switch to Jenkins, I can only help to fix buildbot issues ;-) My point is that with the current tools, it's hard to distinguish regressions to "known failures": buildbots failing since many months. Victor
I created http://bugs.python.org/issue29972 to skip tests known to
fail on AIX. I already proposed two different pull requests which fix
some tests (not all tests yet).
Victor
2017-04-03 16:12 GMT+02:00 Victor Stinner
2017-04-03 16:00 GMT+02:00 David Edelsohn
: I have fixed the Git problem. I thought that it had been working.
Thank you :-) I see tests running: http://buildbot.python.org/all/builders/PPC64%20AIX%203.x/builds/567/steps/t...
The testsuite failures on AIX are issues with the AIX kernel and C Library, often corner cases. I don't want to get into arguments about the POSIX standard. Some of the issues are actual conformance issues and some are different interpretations of the standard.
Many tests of the CPython test suite actually test also the OS, not only Python. But in many cases, it's complex to "mock" the OS (unittest.mock), since tested functions are thin wrapper to syscals.
Addressing the problems in AIX is a slow process. If the failing testcases are too annoying, I would recommend to skip the testcases.
So yeah, let's skip such tests on AIX!
Despite the testsuite failures, Python builds and runs on AIX for the vast majority of users and applications. I don't see the benefit in dropping support for a platform that functions because it doesn't fully pass the testsuite.
It's a limitation of buildbot, the output is basically binary: pass (green) or fail (red). If buildbot would understand test results (parse stdout, or Python produces a output file easier to parse), it would be possible to detect *regressions* compared to previous runs. Using Jenkins and JUnit, I know that it's possible to do that. But I'm not volunteer to handle buildbot or switch to Jenkins, I can only help to fix buildbot issues ;-)
My point is that with the current tools, it's hard to distinguish regressions to "known failures": buildbots failing since many months.
Victor
participants (2)
-
David Edelsohn
-
Victor Stinner