Finding buildbot failures

What's the easiest way of finding which tests failed on buildbot builds? I mean, is there anything easier than using the Web interface to browse to failing builds and then looking at the stdio output in a browser? Thanks, Vinay Sajip

On 25/02/2011 18:10, Vinay Sajip wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 25/02/2011 19:00, exarkun@twistedmatrix.com wrote:
That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance: https://launchpad.net/pyjunitxml I tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything? All the best, Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 01:16 pm, fuzzyman@voidspace.org.uk wrote:
I think this is the relevant pages in the buildbot manual for custom reporting: http://buildbot.net/buildbot/docs/latest/BuildStep-LogFiles.html #BuildStep-LogFiles There's also http://buildbot.net/buildbot/docs/latest/SubunitShellCommand.html#SubunitShe... which is basically the same idea as junitxml, but not actually junitxml itself, so I'd probably go that way. However if you really need junitxml for something, there are tools for converting between subunit and junitxml. Jean-Paul

On 26/02/2011 13:46, exarkun@twistedmatrix.com wrote:
Thanks Jean-Paul. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 28/02/2011 21:59, "Martin v. Löwis" wrote:
To whom was this addressed? If to me I have the desire but lack the skill. If Jean-Paul is willing / able to help and is around for the pycon sprints then it would be a good opportunity. All the best, Michael Foord
Regards, Martin
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

Am 05.03.2011 19:26, schrieb Michael Foord:
I honestly believe that you do have the skill, but perhaps lack the knowledge (and yes, I meant to suggest that you work on this). I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon. Regards, Martin

On 05/03/2011 22:18, "Martin v. Löwis" wrote:
I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon.
Sprints will be an ideal opportunity. All the best, Michael
Regards, Martin
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 25.02.2011 19:10, Vinay Sajip wrote:
Once every failure sent a mail to python-checkins; I had modified the email reporting to include as much info about the failed test as possible. However, there were so many of these emails that we discontinued sending them. Georg

On 25/02/2011 20.10, Vinay Sajip wrote:
You can try bbreport (http://code.google.com/p/bbreport/wiki/Screenshots): hg clone https://bbreport.googlecode.com/hg/ bbreport cd bbreport python bbreport --help python bbreport 3.x (There is some issue with hg revision numbers that I haven't fixed yet, but the above command should work fine) Best Regards, Ezio Melotti

Ezio Melotti <ezio.melotti <at> gmail.com> writes:
Thanks, Ezio, that's really handy! Just what I needed. Example output (for those who haven't used the tool) is at https://gist.github.com/845082 Regards, Vinay Sajip

On 25/02/2011 18:10, Vinay Sajip wrote:
That's one of the big advantages that Jenkins (nee Hudson) has over buildbot - drilling down into individual test failures through the web ui. Your test run needs to generate appropriate xml for that to work though. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 25/02/2011 19:00, exarkun@twistedmatrix.com wrote:
That would be a great improvement to the Python buildbot infrastructure. (Probably a bit small for a GSOC project on its own.) I've had great success using pyjunitxml to generate Hudson compatible reports from unittest test runs (extremely easy to use), and if buildbot *can* handle junit xml format reports then it may be the path of least resistance: https://launchpad.net/pyjunitxml I tried searching (both google and the buildbot docs) for ways to generate more complete reports or to integrate junitxml with builbot, but failed. Can you point me at anything? All the best, Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 01:16 pm, fuzzyman@voidspace.org.uk wrote:
I think this is the relevant pages in the buildbot manual for custom reporting: http://buildbot.net/buildbot/docs/latest/BuildStep-LogFiles.html #BuildStep-LogFiles There's also http://buildbot.net/buildbot/docs/latest/SubunitShellCommand.html#SubunitShe... which is basically the same idea as junitxml, but not actually junitxml itself, so I'd probably go that way. However if you really need junitxml for something, there are tools for converting between subunit and junitxml. Jean-Paul

On 26/02/2011 13:46, exarkun@twistedmatrix.com wrote:
Thanks Jean-Paul. Michael
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 28/02/2011 21:59, "Martin v. Löwis" wrote:
To whom was this addressed? If to me I have the desire but lack the skill. If Jean-Paul is willing / able to help and is around for the pycon sprints then it would be a good opportunity. All the best, Michael Foord
Regards, Martin
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

Am 05.03.2011 19:26, schrieb Michael Foord:
I honestly believe that you do have the skill, but perhaps lack the knowledge (and yes, I meant to suggest that you work on this). I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon. Regards, Martin

On 05/03/2011 22:18, "Martin v. Löwis" wrote:
I'd be also willing to help you getting started, especially assuming you would want to sprint on that during PyCon.
Sprints will be an ideal opportunity. All the best, Michael
Regards, Martin
-- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html

On 25.02.2011 19:10, Vinay Sajip wrote:
Once every failure sent a mail to python-checkins; I had modified the email reporting to include as much info about the failed test as possible. However, there were so many of these emails that we discontinued sending them. Georg

On 25/02/2011 20.10, Vinay Sajip wrote:
You can try bbreport (http://code.google.com/p/bbreport/wiki/Screenshots): hg clone https://bbreport.googlecode.com/hg/ bbreport cd bbreport python bbreport --help python bbreport 3.x (There is some issue with hg revision numbers that I haven't fixed yet, but the above command should work fine) Best Regards, Ezio Melotti

Ezio Melotti <ezio.melotti <at> gmail.com> writes:
Thanks, Ezio, that's really handy! Just what I needed. Example output (for those who haven't used the tool) is at https://gist.github.com/845082 Regards, Vinay Sajip
participants (7)
-
"Martin v. Löwis"
-
Antoine Pitrou
-
exarkun@twistedmatrix.com
-
Ezio Melotti
-
Georg Brandl
-
Michael Foord
-
Vinay Sajip