[core-workflow] Use buildbot to test tracker patches

[Ezio Melotti]

On Mon, Jul 7, 2014 at 4:55 PM, Antoine Pitrou <antoine at python.org> wrote:
>
> Hi,
>
> Le 05/07/2014 08:46, Ezio Melotti a écrit :
>
>> I recently had a chance to learn a bit more about buildbot, and it
>> occurred to me than adding a new builder for testing tracker patches
>> shouldn't be too difficult.
>>
>> This is how it should work:
>>    1) a new "Test with buildbot" (or whatever) button is added to the
>> tracker next to each patch, and developers/triagers can press it to
>> trigger the build;
>>    2) the buildbot builder receives the URL of the issue and of the
>> patch and run all the usual steps with two differences:
>>      a) a new step to run "hg import --no-c url_of_the_patch" is added
>> after the cloning step;
>>      b) an email with the results is sent back to the bug tracker once
>> the slaves are done;
>
>
> I guess the question is: which problem is being solved?
>
> - have tests run faster - that's unlikely, unless we get a very fast builder
> - have tests run more reliably - unlikely as well, IMO
> - run tests on a different environment - that's useful; but that assumes we
> have a couple different builders which are fast, reliable and with a
> diversity of OSes
>

The main problem that is being solved is avoiding to manually download
a patch, apply/test it, and report the results on the tracker.  The
ability to easily test a patch on different OSes is also very useful
(lot of times I've seen windows buildbots failing after
committing/pushing a patch tested on linux only).
Checking if a patch (still) applies could also be done by the tracker
itself without involving buildbot, but seeing if the tests (still)
pass, and if they pass on different platforms requires buildbot.

> Regards
>
> Antoine.
>
>
>
> _______________________________________________
> core-workflow mailing list
> core-workflow at python.org
> https://mail.python.org/mailman/listinfo/core-workflow
> This list is governed by the PSF Code of Conduct:
> https://www.python.org/psf/codeofconduct