Presentation on Rust's GitHub based automation tools
I was at linux.conf.au 2016 last week, and one of the presentations was from Mozilla's Emily Dunham on some of the infrastructure automation they use with Rust and other GitHub based projects: https://www.youtube.com/watch?v=dIageYT0Vgg In addition to their merge bot project homu (which we've talked about previously), they also have: highfive (a greeter bot): https://github.com/nrc/highfive starters (an issue curator): https://starters.servo.org/ The first project looks for folks submitting their first PR or issue, and responds with some standard info to save humans from having to do it later (like pointers to the Code of Conduct and the Contributor Licensing Agreement, as well as explanations of how the contribution process works) The second one is designed to provide a better answer to the "How do I get started?" question by making it easy for developers to tag simpler issues. This could presumably also be used to provide separate views based on what folks want to work on (e.g. documentation, Python code, C code) While these wouldn't necessarily be something we wanted to set up immediately, it likely makes a lot of sense to try to share the tool maintenance load with Mozilla rather than going for a completely custom setup. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Sun, Feb 7, 2016 at 9:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I was at linux.conf.au 2016 last week, and one of the presentations was from Mozilla's Emily Dunham on some of the infrastructure automation they use with Rust and other GitHub based projects: https://www.youtube.com/watch?v=dIageYT0Vgg
I just watched it, great talk. Thanks for sharing!
In addition to their merge bot project homu (which we've talked about previously), they also have:
highfive (a greeter bot): https://github.com/nrc/highfive
This is a good idea.
starters (an issue curator): https://starters.servo.org/
While these wouldn't necessarily be something we wanted to set up immediately, it likely makes a lot of sense to try to share the tool maintenance load with Mozilla rather than going for a completely custom setup.
The biggest problem of these tools is that they don't provide an API or a framework to use as a base. They have a lot of project specific code and they don't work on Python 3. So you'll need to write your own code anyway. We are going to write a lot of bots in the next months so I think we will eventually create some sort of framework to share some code. Coordinating with Mozilla (or any other organization) requires a big amount of time, and I simply don't have enough time and motivation right now. However, I'm planning to send an email to the django-developers list [1] when I finish to document my bot. I have a test organization at https://github.com/fayton. See also https://github.com/fayton/cpython/pull/1 for an example pull request (the name of the bot is just a placeholder, Brett will give it a name :)) --Berker [1] They might be interested since we (will) have almost identical workflow with them (they also have multiple maintenance branches for example)
On Sun, Feb 7, 2016 at 10:14 AM, Berker Peksağ <berker.peksag@gmail.com> wrote:
On Sun, Feb 7, 2016 at 9:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I was at linux.conf.au 2016 last week, and one of the presentations was from Mozilla's Emily Dunham on some of the infrastructure automation they use with Rust and other GitHub based projects: https://www.youtube.com/watch?v=dIageYT0Vgg
I just watched it, great talk. Thanks for sharing!
In addition to their merge bot project homu (which we've talked about previously), they also have:
highfive (a greeter bot): https://github.com/nrc/highfive
This is a good idea.
starters (an issue curator): https://starters.servo.org/
While these wouldn't necessarily be something we wanted to set up immediately, it likely makes a lot of sense to try to share the tool maintenance load with Mozilla rather than going for a completely custom setup.
The biggest problem of these tools is that they don't provide an API or a framework to use as a base. They have a lot of project specific code and they don't work on Python 3. So you'll need to write your own code anyway. We are going to write a lot of bots in the next months so I think we will eventually create some sort of framework to share some code.
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
Coordinating with Mozilla (or any other organization) requires a big amount of time, and I simply don't have enough time and motivation right now. However, I'm planning to send an email to the django-developers list [1] when I finish to document my bot. I have a test organization at https://github.com/fayton. See also https://github.com/fayton/cpython/pull/1 for an example pull request (the name of the bot is just a placeholder, Brett will give it a name :))
--Berker
[1] They might be interested since we (will) have almost identical workflow with them (they also have multiple maintenance branches for example) _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
Maciej
On Sun, 7 Feb 2016 at 02:23 Maciej Szulik <soltysh@gmail.com> wrote:
On Sun, Feb 7, 2016 at 10:14 AM, Berker Peksağ <berker.peksag@gmail.com> wrote:
On Sun, Feb 7, 2016 at 9:21 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
I was at linux.conf.au 2016 last week, and one of the presentations was from Mozilla's Emily Dunham on some of the infrastructure automation they use with Rust and other GitHub based projects: https://www.youtube.com/watch?v=dIageYT0Vgg
I just watched it, great talk. Thanks for sharing!
In addition to their merge bot project homu (which we've talked about previously), they also have:
highfive (a greeter bot): https://github.com/nrc/highfive
The greeter bit could be rolled into https://github.com/brettcannon/knights-who-say-ni since it has to watch for newly opened pull requests for the CLA check anyway.
This is a good idea.
starters (an issue curator): https://starters.servo.org/
While these wouldn't necessarily be something we wanted to set up immediately, it likely makes a lot of sense to try to share the tool maintenance load with Mozilla rather than going for a completely custom setup.
The biggest problem of these tools is that they don't provide an API or a framework to use as a base. They have a lot of project specific code and they don't work on Python 3. So you'll need to write your own code anyway. We are going to write a lot of bots in the next months so I think we will eventually create some sort of framework to share some code.
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
Having started writing the CLA bot, I can attest that it's tough to abstract it all in a way that's easy to swap out the parts. I am trying to do my best, though, so that when it comes to swap out either the server host, contribution host, or CLA records host it won't be a complete rewrite. -Brett
Coordinating with Mozilla (or any other organization) requires a big amount of time, and I simply don't have enough time and motivation right now. However, I'm planning to send an email to the django-developers list [1] when I finish to document my bot. I have a test organization at https://github.com/fayton. See also https://github.com/fayton/cpython/pull/1 for an example pull request (the name of the bot is just a placeholder, Brett will give it a name :))
--Berker
[1] They might be interested since we (will) have almost identical workflow with them (they also have multiple maintenance branches for example) _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
Maciej _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On 7 February 2016 at 20:23, Maciej Szulik <soltysh@gmail.com> wrote:
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
It was a combination of a suggestion and a question. The suggestion was "Rust's automation UX seems nice, I think it would be desirable to target similar capabilities for CPython", the question was "Would it be feasible to collaborate on actual automation development?". It sounds like the pragmatic answer to the latter is "No, the additional coordination overhead isn't worth the potential pay-off", and I think that's fine - our respective communities can still learn from each other when it comes to our definitions of "What does 'good' look like?" in workflow design. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Mon, Feb 8, 2016 at 5:07 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 7 February 2016 at 20:23, Maciej Szulik <soltysh@gmail.com> wrote:
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
It was a combination of a suggestion and a question. The suggestion was "Rust's automation UX seems nice, I think it would be desirable to target similar capabilities for CPython", the question was "Would it be feasible to collaborate on actual automation development?".
It sounds like the pragmatic answer to the latter is "No, the additional coordination overhead isn't worth the potential pay-off", and I think that's fine - our respective communities can still learn from each other when it comes to our definitions of "What does 'good' look like?" in workflow design.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I've also reminded one really handy solution described in the presentation, which is auto-assigning PR to the owner of certain area. With the list we keep here: https://docs.python.org/devguide/experts.html we could pretty easily do such mechanism. This will be handy for the devs because assigning a specific issue will trigger an email notification of such, which in turn is similar to our noisy in bug tracker. Otherwise the PRs might end up hanging there until somebody will do that manually. Having said that Brett if you need help with it - I'm here to help you. Cheers, Maciej
On Mon, 8 Feb 2016 at 13:11 Maciej Szulik <soltysh@gmail.com> wrote:
On Mon, Feb 8, 2016 at 5:07 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 7 February 2016 at 20:23, Maciej Szulik <soltysh@gmail.com> wrote:
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
It was a combination of a suggestion and a question. The suggestion was "Rust's automation UX seems nice, I think it would be desirable to target similar capabilities for CPython", the question was "Would it be feasible to collaborate on actual automation development?".
It sounds like the pragmatic answer to the latter is "No, the additional coordination overhead isn't worth the potential pay-off", and I think that's fine - our respective communities can still learn from each other when it comes to our definitions of "What does 'good' look like?" in workflow design.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I've also reminded one really handy solution described in the presentation, which is auto-assigning PR to the owner of certain area. With the list we keep here: https://docs.python.org/devguide/experts.html we could pretty easily do such mechanism. This will be handy for the devs because assigning a specific issue will trigger an email notification of such, which in turn is similar to our noisy in bug tracker. Otherwise the PRs might end up hanging there until somebody will do that manually.
Having said that Brett if you need help with it - I'm here to help you.
:) Thanks. Once we have migrated the repositories over we can start discussing enhancements to the workflow like automatic reviewer assignment (and I personally have some ideas about PR assignment as well for when there isn't an expert). But I don't want to get too distracted by this bonus work when we haven't even started most of the work required to simply match our current workflow. It's great that people are excited about making things better and I don't want to squash people's energy to help, but I also don't want to get too distracted by enhancements when we haven't even started a bunch of the minimum work to even move to GitHub, let alone take advantage of what bonus features it will bring to the table. My current worry is that we are going to get blocked on Roundup work because right now only Ezio and R. David know how that stuff works. Once the CLA bot is finished I'm going to shift to helping with that, but it will obviously go faster if others can also help with bugs.python.org work because we can't switch over until we have a minimum workflow that matches our current one.
On Tue, Feb 9, 2016 at 7:54 PM, Brett Cannon <brett@python.org> wrote:
On Mon, 8 Feb 2016 at 13:11 Maciej Szulik <soltysh@gmail.com> wrote:
On Mon, Feb 8, 2016 at 5:07 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
On 7 February 2016 at 20:23, Maciej Szulik <soltysh@gmail.com> wrote:
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
It was a combination of a suggestion and a question. The suggestion was "Rust's automation UX seems nice, I think it would be desirable to target similar capabilities for CPython", the question was "Would it be feasible to collaborate on actual automation development?".
It sounds like the pragmatic answer to the latter is "No, the additional coordination overhead isn't worth the potential pay-off", and I think that's fine - our respective communities can still learn from each other when it comes to our definitions of "What does 'good' look like?" in workflow design.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I've also reminded one really handy solution described in the presentation, which is auto-assigning PR to the owner of certain area. With the list we keep here: https://docs.python.org/devguide/experts.html we could pretty easily do such mechanism. This will be handy for the devs because assigning a specific issue will trigger an email notification of such, which in turn is similar to our noisy in bug tracker. Otherwise the PRs might end up hanging there until somebody will do that manually.
Having said that Brett if you need help with it - I'm here to help you.
:) Thanks. Once we have migrated the repositories over we can start discussing enhancements to the workflow like automatic reviewer assignment (and I personally have some ideas about PR assignment as well for when there isn't an expert).
But I don't want to get too distracted by this bonus work when we haven't even started most of the work required to simply match our current workflow. It's great that people are excited about making things better and I don't want to squash people's energy to help, but I also don't want to get too distracted by enhancements when we haven't even started a bunch of the minimum work to even move to GitHub, let alone take advantage of what bonus features it will bring to the table.
My current worry is that we are going to get blocked on Roundup work because right now only Ezio and R. David know how that stuff works. Once the CLA bot is finished I'm going to shift to helping with that, but it will obviously go faster if others can also help with bugs.python.org work because we can't switch over until we have a minimum workflow that matches our current one.
What's there to be done and how can I help with that, in that case? Maciej
See https://mail.python.org/pipermail/core-workflow/2016-February/000469.html and Ezio's follow-up (and yes, you can help :) . On Wed, 10 Feb 2016 at 04:29 Maciej Szulik <soltysh@gmail.com> wrote:
On Tue, Feb 9, 2016 at 7:54 PM, Brett Cannon <brett@python.org> wrote:
On Mon, 8 Feb 2016 at 13:11 Maciej Szulik <soltysh@gmail.com> wrote:
On Mon, Feb 8, 2016 at 5:07 AM, Nick Coghlan <ncoghlan@gmail.com>
On 7 February 2016 at 20:23, Maciej Szulik <soltysh@gmail.com> wrote:
Talking from the position of owning a similar bot in OpenShift, I quite certain that it's really hard to have common base. Since these bots address specific project and there are not two exactly the same projects with exactly the same workflow. I think what Nick meant to show is, what we should target, more or less at least.
It was a combination of a suggestion and a question. The suggestion was "Rust's automation UX seems nice, I think it would be desirable to target similar capabilities for CPython", the question was "Would it be feasible to collaborate on actual automation development?".
It sounds like the pragmatic answer to the latter is "No, the additional coordination overhead isn't worth the potential pay-off", and I think that's fine - our respective communities can still learn from each other when it comes to our definitions of "What does 'good' look like?" in workflow design.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I've also reminded one really handy solution described in the presentation, which is auto-assigning PR to the owner of certain area. With the list we keep here: https://docs.python.org/devguide/experts.html we could pretty easily do such mechanism. This will be handy for the devs because assigning a specific issue will trigger an email notification of such, which in turn is similar to our noisy in bug tracker. Otherwise the PRs might end up hanging there until somebody will do that manually.
Having said that Brett if you need help with it - I'm here to help you.
:) Thanks. Once we have migrated the repositories over we can start discussing enhancements to the workflow like automatic reviewer assignment (and I personally have some ideas about PR assignment as well for when
wrote: there
isn't an expert).
But I don't want to get too distracted by this bonus work when we haven't even started most of the work required to simply match our current workflow. It's great that people are excited about making things better and I don't want to squash people's energy to help, but I also don't want to get too distracted by enhancements when we haven't even started a bunch of the minimum work to even move to GitHub, let alone take advantage of what bonus features it will bring to the table.
My current worry is that we are going to get blocked on Roundup work because right now only Ezio and R. David know how that stuff works. Once the CLA bot is finished I'm going to shift to helping with that, but it will obviously go faster if others can also help with bugs.python.org work because we can't switch over until we have a minimum workflow that matches our current one.
What's there to be done and how can I help with that, in that case?
Maciej
On Sat, 6 Feb 2016 at 23:21 Nick Coghlan <ncoghlan@gmail.com> wrote:
I was at linux.conf.au 2016 last week, and one of the presentations was from Mozilla's Emily Dunham on some of the infrastructure automation they use with Rust and other GitHub based projects: https://www.youtube.com/watch?v=dIageYT0Vgg
I finally had the time to watch the video. It's very interesting and basically mirrors what I'm hoping is going to be possible for us once we migrate to GitHub (i.e., automate as much as we can). While it doesn't change any of the plans for reaching parity support to the hg workflow, it is something to revisit when it comes time to look at adding new features to our workflow (and maybe even share the bots if it happens to make sense). -Brett
In addition to their merge bot project homu (which we've talked about previously), they also have:
highfive (a greeter bot): https://github.com/nrc/highfive starters (an issue curator): https://starters.servo.org/
The first project looks for folks submitting their first PR or issue, and responds with some standard info to save humans from having to do it later (like pointers to the Code of Conduct and the Contributor Licensing Agreement, as well as explanations of how the contribution process works)
The second one is designed to provide a better answer to the "How do I get started?" question by making it easy for developers to tag simpler issues. This could presumably also be used to provide separate views based on what folks want to work on (e.g. documentation, Python code, C code)
While these wouldn't necessarily be something we wanted to set up immediately, it likely makes a lot of sense to try to share the tool maintenance load with Mozilla rather than going for a completely custom setup.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
participants (4)
-
Berker Peksağ -
Brett Cannon -
Maciej Szulik -
Nick Coghlan