OpenBSD buildbot has many failures
Hi, The OpenBSD buildbot always fail with the same failures since many months (ex: test_socket). Is someone interested to fix them? If no, would it be possible to turn it off to get a better view of regressions? Currently, they are too many red buildbots to quickly see regressions introduced by recent commits. http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x Victor
Hi Victor — I am personally interested in seeing all tests pass on OpenBSD and am willing to put forth effort to help that be so. I would be happy to be added to any issues that get opened against OpenBSD. That said, I have concerns about the nature of when and how these failures came about — specifically I worry that other devs have committed the changes which prompted these failures yet they did not pay attention nor take responsibility when it happened. Having monitored certain buildbots for a while to see how the community behaves and devs fail to react when a failure is triggered by a commit, I think we should do much better in taking individual responsibility for prompting these failures. Davin
On Mar 28, 2015, at 04:53, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
The OpenBSD buildbot always fail with the same failures since many months (ex: test_socket). Is someone interested to fix them? If no, would it be possible to turn it off to get a better view of regressions? Currently, they are too many red buildbots to quickly see regressions introduced by recent commits.
http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x
Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/python%2Bpython_dev%40dis...
I, too, would be interested in having tests pass on OpenBSD (and NetBSD) and am willing to do whatever I have to to make it be so. -- H On 31 March 2015 at 21:52, Davin Potts <python@discontinuity.net> wrote:
Hi Victor —
I am personally interested in seeing all tests pass on OpenBSD and am willing to put forth effort to help that be so. I would be happy to be added to any issues that get opened against OpenBSD. That said, I have concerns about the nature of when and how these failures came about — specifically I worry that other devs have committed the changes which prompted these failures yet they did not pay attention nor take responsibility when it happened. Having monitored certain buildbots for a while to see how the community behaves and devs fail to react when a failure is triggered by a commit, I think we should do much better in taking individual responsibility for prompting these failures.
Davin
On Mar 28, 2015, at 04:53, Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,
The OpenBSD buildbot always fail with the same failures since many months (ex: test_socket). Is someone interested to fix them? If no, would it be possible to turn it off to get a better view of regressions? Currently, they are too many red buildbots to quickly see regressions introduced by recent commits.
http://buildbot.python.org/all/builders/x86%20OpenBSD%205.5%203.x
Victor _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/python%2Bpython_dev%40dis...
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/hasan.diwan%40gmail.com
-- OpenPGP: https://hasan.d8u.us/gpg.key Sent from my mobile device Envoyé de mon portable
Le mercredi 1 avril 2015, Davin Potts <python@discontinuity.net> a écrit :
I am personally interested in seeing all tests pass on OpenBSD and am willing to put forth effort to help that be so.
Great!
I would be happy to be added to any issues that get opened against OpenBSD.
You can do a search in bugs.python.org. If I recall recorrectly, there are only a few open issues.
That said, I have concerns about the nature of when and how these failures came about — specifically I worry that other devs have committed the changes which prompted these failures yet they did not pay attention nor take responsibility when it happened.
That's the purpose of my previous email. If the buildbot is always red, nobody will check it anymore. I'm quite sure that the current OpenBSD 5.5 was always red since its setup. Sometimes, I try to fix some isssues. Fixing OpenBSD issues usually means install OpenBSD at home because most issues are specific to OpenBSD. Even for LibreSSL, OpenBSD behaves differently (on the version number) than FreeBSD. Having monitored certain buildbots for a while to see how the community
behaves and devs fail to react when a failure is triggered by a commit, I think we should do much better in taking individual responsibility for prompting these failures.
When I started to work on Python, I was surprised to not get emails from buildbots. There are different issues. Dome buildbots are always red. Threre are still many sporadic issues: temporary network failures seen as bugs, threads issues. Multiprocessing issues. Recently, I saw some rare asyncio issues. So we reduce the number of these issues before spaming developers. Currently, there is an IRC bot on #python-dev which notify when buildbot color changes. Or sometiles I chceck the huge waterfall page. By the way, it became difficult to browse this page because there are too many offline buildbots. Victor
On 01/04/2015 08:32, Victor Stinner wrote:
When I started to work on Python, I was surprised to not get emails from buildbots.
Currently, there is an IRC bot on #python-dev which notify when buildbot color changes. Or sometiles I chceck the huge waterfall page. By the way, it became difficult to browse this page because there are too many offline buildbots.
On the back of Victor's recent emails re buildbots, I've knocked something up which can be scheduled to email the status of some or all buildbots: https://github.com/tjguk/buildbotinfo It's rough-and-ready but it does work. If it's useful to anyone, feel free to use / clone / fork / whatever. By all means send PRs or raise Issues but I've already overrun the little time I'd given myself to get this working so I'm not sure when I'll get to them. There are two modules: the underlying buildbot.py which lightly wraps the XML-RPC interface; and the buildbotinfo.py which uses it to generate some readable output according to some selection criteria. See the project README for some details: https://github.com/tjguk/buildbotinfo/blob/master/README.rst TJG
Hi, 2015-04-01 12:47 GMT+02:00 Tim Golden <mail@timgolden.me.uk>:
On the back of Victor's recent emails re buildbots, I've knocked something up which can be scheduled to email the status of some or all buildbots:
Are you aware of this previous project? https://code.google.com/p/bbreport/ I also wrote two very simple scripts to download and parse buildbot output: https://bitbucket.org/haypo/misc/src/5929cc110f0352cecb384adceae3647f26fa693... https://bitbucket.org/haypo/misc/src/5929cc110f0352cecb384adceae3647f26fa693... I'm more interested on the parser part. My first goal was to compute a summary in one line: success, timeout, fatal error, etc. The next step would be to parse test failures to be able to ignore tests known to fail, to only detect regressions. It would be better than the current "red" / "green" status. For example, I want to always ignore MemoryError just because I'm unable to fix buildbots which are known to regulary fail with MemoryError (ex: OpenIndiana). I used Jenkins which uses JUnit reports. It's much more powerful because it computes statistics on tests to check which tests fail randomly, which tests are new failures, etc. I don't think that it would be complex to generate a JUnit report (or another easy to parse report) in regrtest. But XML is maybe not the best format to handle timeout and fatal errors :-) Victor
On 01/04/2015 13:13, Victor Stinner wrote:
Hi,
2015-04-01 12:47 GMT+02:00 Tim Golden <mail@timgolden.me.uk>:
On the back of Victor's recent emails re buildbots, I've knocked something up which can be scheduled to email the status of some or all buildbots:
Are you aware of this previous project? https://code.google.com/p/bbreport/
Yes -- I originally forked it with a view to extending it the way I wanted, but I was rewriting so much that in the end I started afresh and ended up with just my own code. (The way you do...) [... snip useful ideas about more finely-tuned results ...] I'd also thought of various directions things could go. TBH, though, I knew if I started to get ambitious I'd end up with something which tried to do everything and didn't do anything. I might go somewhere with the thing in the future, but for now it does what it does and if it helps someone -- including me -- at the simplest level, well that's a good thing. TJG
On 1 April 2015 at 17:32, Victor Stinner <victor.stinner@gmail.com> wrote:
Currently, there is an IRC bot on #python-dev which notify when buildbot color changes. Or sometiles I chceck the huge waterfall page. By the way, it became difficult to browse this page because there are too many offline buildbots.
The categorised links at https://www.python.org/dev/buildbot/ can be a bit more usable (but the stable links naturally exclude the systems we've never got working reliably in the first place). An idea that's been kicking around for a while has been to start using BuildBot's ephemeral test client support to kick off fresh VMs in Rackspace for x86_64 testing on the OS versions available there rather than relying solely on donated test systems. Unfortunately there are enough unknowns involved in that that we don't even know how hard its likely to be to set up without someone being in a position to contribute the time to start figuring it out. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 01.04.15 07:52, Davin Potts wrote:
I am personally interested in seeing all tests pass on OpenBSD and am willing to put forth effort to help that be so. I would be happy to be added to any issues that get opened against OpenBSD. That said, I have concerns about the nature of when and how these failures came about — specifically I worry that other devs have committed the changes which prompted these failures yet they did not pay attention nor take responsibility when it happened. Having monitored certain buildbots for a while to see how the community behaves and devs fail to react when a failure is triggered by a commit, I think we should do much better in taking individual responsibility for prompting these failures.
participants (6)
-
Davin Potts -
Hasan Diwan -
Nick Coghlan -
Serhiy Storchaka -
Tim Golden -
Victor Stinner