What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
Thanks,
Vinay Sajip
On Fri, 25 Feb 2011 18:10:28 +0000 (UTC) Vinay Sajip vinay_sajip@yahoo.co.uk wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
Not really, but that becomes quite easy once you're used to it. Good luck :)
Regards
Antoine.
On 25/02/2011 18:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Michael
Thanks,
Vinay Sajip
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On 06:47 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 18:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
Jean-Paul
On 25/02/2011 19:00, exarkun@twistedmatrix.com wrote:
On 06:47 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 18:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance:
https://launchpad.net/pyjunitxml
I tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything?
All the best,
Michael
Jean-Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
On 01:16 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 19:00, exarkun@twistedmatrix.com wrote:
On 06:47 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 18:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance:
https://launchpad.net/pyjunitxmlI tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything?
I think this is the relevant pages in the buildbot manual for custom reporting:
http://buildbot.net/buildbot/docs/latest/BuildStep-LogFiles.html #BuildStep-LogFiles
There's also http://buildbot.net/buildbot/docs/latest/SubunitShellCommand.html#SubunitShe...
which is basically the same idea as junitxml, but not actually junitxml itself, so I'd probably go that way. However if you really need junitxml for something, there are tools for converting between subunit and junitxml.
Jean-Paul
On 26/02/2011 13:46, exarkun@twistedmatrix.com wrote:
On 01:16 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 19:00, exarkun@twistedmatrix.com wrote:
On 06:47 pm, fuzzyman@voidspace.org.uk wrote:
On 25/02/2011 18:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance:
https://launchpad.net/pyjunitxmlI tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything?
I think this is the relevant pages in the buildbot manual for custom reporting:
http://buildbot.net/buildbot/docs/latest/BuildStep-LogFiles.html #BuildStep-LogFiles
There's also http://buildbot.net/buildbot/docs/latest/SubunitShellCommand.html#SubunitShe...
which is basically the same idea as junitxml, but not actually junitxml itself, so I'd probably go that way. However if you really need junitxml for something, there are tools for converting between subunit and junitxml.
Thanks Jean-Paul.
Michael
Jean-Paul _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.u...
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure.
So would you be willing to contribute the necessary changes?
Regards, Martin
On 28/02/2011 21:59, "Martin v. Löwis" wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure.
So would you be willing to contribute the necessary changes?
To whom was this addressed? If to me I have the desire but lack the skill. If Jean-Paul is willing / able to help and is around for the pycon sprints then it would be a good opportunity.
All the best,
Michael Foord
Regards, Martin
On Sat, 05 Mar 2011 18:26:29 +0000 Michael Foord fuzzyman@voidspace.org.uk wrote:
On 28/02/2011 21:59, "Martin v. Löwis" wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure.
So would you be willing to contribute the necessary changes?
To whom was this addressed? If to me I have the desire but lack the skill.
It shouldn't be difficult to acquire the skills. Buildbot is pure Python (even its configuration file).
Regards
Antoine.
Am 05.03.2011 19:26, schrieb Michael Foord:
On 28/02/2011 21:59, "Martin v. Löwis" wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure.
So would you be willing to contribute the necessary changes?
To whom was this addressed? If to me I have the desire but lack the skill.
I honestly believe that you do have the skill, but perhaps lack the knowledge (and yes, I meant to suggest that you work on this).
I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon.
Regards, Martin
On 05/03/2011 22:18, "Martin v. Löwis" wrote:
Am 05.03.2011 19:26, schrieb Michael Foord:
On 28/02/2011 21:59, "Martin v. Löwis" wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though.
Buildbot can do this too. It can even do it without xml, although it does need *some* parseable format, which I think the Python test suite is a long way from.
That would be a great improvement to the Python buildbot infrastructure.
So would you be willing to contribute the necessary changes?
To whom was this addressed? If to me I have the desire but lack the skill.
I honestly believe that you do have the skill, but perhaps lack the knowledge (and yes, I meant to suggest that you work on this).
Yes, that's what I meant.
I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon.
Sprints will be an ideal opportunity.
All the best,
Michael
Regards, Martin
On 25.02.2011 19:10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
Once every failure sent a mail to python-checkins; I had modified the email reporting to include as much info about the failed test as possible.
However, there were so many of these emails that we discontinued sending them.
Georg
On 25/02/2011 20.10, Vinay Sajip wrote:
What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser?
You can try bbreport (http://code.google.com/p/bbreport/wiki/Screenshots):
hg clone https://bbreport.googlecode.com/hg/ bbreport cd bbreport python bbreport --help python bbreport 3.x
(There is some issue with hg revision numbers that I haven't fixed yet, but the above command should work fine)
Best Regards, Ezio Melotti
Thanks,
Vinay Sajip
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ezio.melotti%40gmail.com
Ezio Melotti <ezio.melotti <at> gmail.com> writes:
You can try bbreport (http://code.google.com/p/bbreport/wiki/Screenshots):
hg clone https://bbreport.googlecode.com/hg/ bbreport cd bbreport python bbreport --help python bbreport 3.x
(There is some issue with hg revision numbers that I haven't fixed yet, but the above command should work fine)
Thanks, Ezio, that's really handy! Just what I needed. Example output (for those who haven't used the tool) is at
https://gist.github.com/845082
Regards,
Vinay Sajip
-
"Martin v. Löwis" -
Antoine Pitrou -
exarkun＠twistedmatrix.com -
Ezio Melotti -
Georg Brandl -
Michael Foord -
Vinay Sajip