[core-workflow] Use buildbot to test tracker patches

[anatoly techtonik]

On Sat, Jul 5, 2014 at 9:30 PM, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 05/07/2014 13:46, Ezio Melotti wrote:
>>
>> Hi,
>> I recently had a chance to learn a bit more about buildbot, and it
>> occurred to me than adding a new builder for testing tracker patches
>> shouldn't be too difficult.
>>
>> This is how it should work:
>>    1) a new "Test with buildbot" (or whatever) button is added to the
>> tracker next to each patch, and developers/triagers can press it to
>> trigger the build;
>>    2) the buildbot builder receives the URL of the issue and of the
>> patch and run all the usual steps with two differences:
>>      a) a new step to run "hg import --no-c url_of_the_patch" is added
>> after the cloning step;
>>      b) an email with the results is sent back to the bug tracker once
>> the slaves are done;
>>
>> Unless I'm missing something, both these changes should be trivial.
>> On the tracker side is enough to add a form with a couple of (hidden)
>> fields with the URLs and a submit button.  On the buildbot side it
>> should be enough to create a new builder (by copying one of the
>> existing ones in the buildbot config file and adding the two steps
>> mentioned above).  Putting together a mail summary with the results of
>> all the slaves might require a bit of extra code, but it shouldn't be
>> anything too difficult.
>>
>> Some of the issues with this proposal are:
>>    1) users uploading patches with malicious code -- this shouldn't be
>> a problem if we only allow developers/triagers to trigger the build
>> manually and tell them to review the code before doing so;
>>    2) overloading of the slaves -- cloning/compiling/testing/cleaning
>> takes quite a lot of time, and if this new feature starts to be used
>> heavily it might start to give too much work for the slaves.  I don't
>> know what the current load is though, so I'm not sure this will be an
>> issue;
>>    3) testing/implementing the feature -- I think it should be possible
>> to test the feature locally before actually applying the changes on
>> the real buildbot master, but unless someone is going to apply them on
>> my behalf I would probably need access to the buildbot repo (is there
>> a public repo?);
>>    4) determine which branch the patch should be applied to -- this
>> could be done by adding a new "branch" field on the tracker; rietveld
>> could use it too;
>>    5) if the patch fails to apply there's no need to launch the build
>> on all the slaves -- a single slave (or maybe even the master?) could
>> check if the patch applies cleanly, and if it does trigger all the
>> other slaves.  More fine-grained options could be added to the request
>> form too (e.g. run only on specific slaves/operating systems);
>>
>> If the idea is reasonable and I didn't miss anything I can try to find
>> some time to work on it.
>>
>> Best Regards,
>> Ezio Melotti
>
>
> Anything to make life easier for everybody.
>
> I'd offer to help myself, but having seen the name of a well known PITA on
> this thread I simply won't bother, it's just more trouble than it's worth.

If you expect me to say "please, please, please, oh help Ezio", then of
course, I won't say it. But if you send a pull request, I won't be the one who
will decline it, because of emotional reasons. The problem is:

1. There is no place to send PR against
2. There is no workflow
3. There is no roadmap

So if you can help Ezio close any of those problems while he is working on
Roundup and server configs, in other words help him to make it easier for
other people people to help him, that will be a good deal already and a good
ontopic activity. I promise not to deliver any emotional distress in the
process.

> I'm fully aware that I'm no angel, but I can at least plead mitigating
> circumstances.

Nobody is an angel, but "He who fights monsters should see to it that he
himself does not become a monster."
-- 
anatoly t.