My initial thoughts on the steps/blockers of the transition
So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks. The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython. I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos. Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo. First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file. Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org? Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually. Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views. Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub. Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller). Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers. CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it). Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about. Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :). Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of. Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us. Six, we will need to update our Buildbot fleet. This gets us to the bare minimum needed to function. Parity with hg.python.org ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org that the PR exists. If people disagree then feel free to say so. Adding perks ========== Now we get to some added stuff that we never had on our own infrastructure. :) We should wire up CI for all PRs. I don't know if we want to go with Travis, Codeship, or what CI provider, but we should definitely hook it up and fully utilize the resource. This could even include running doctest over the docs, making sure the reST markup is accurate, etc. Do we need to set up a web hook to trigger website rebuilds? We should at least have a mirror on Read the Docs that is triggered by web hook so that we have a backup of the documentation (if possible; not sure how custom our Sphinx setup is compared to what they require to work). We should try to get test coverage wired up as well per CI. I don't know if coveralls.io or some other provider is best, but we should see what is available and find out if we can use them to either get basic coverage or thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124 to see what thorough coverage entails, but it does require a checkout of coverage.py). We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death scenes from the Holy Grail for inspiration since they deal with blocking you from doing something). It should handle specifying the commit message, what branches to commit to/cherry pick into, and a NEWS entry (if necessary). I don't know if it needs to do anything else as a requirement. It should probably implement a commit queue like Zuul or Homu (and both of those can be considered as the basis of the bot). Also gating commits on passing a test run probably would also be good. I'm sure we will want to use some labels and milestones to track what PRs are for what versions, if they are blocked on something, etc. --- Long email! :) I think that is my current brain dumped in email form. As I said at the beginning, I think we should focus on what is blocking the easiest repos first and then just keep knocking down blockers as we try to move over more repos.
On Jan 4, 2016, at 7:42 PM, Brett Cannon
wrote: So consider this the starting discussion of the PEP that will be the hg.python.org http://hg.python.org/ -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython. I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
We should have a team for each logical group of access I think. We can add people to multiple teams, or to just one team. The list of people who can access *all* repos on the Python org is https://caremad.io/s/7Q0wcgOcoE/, everyone else (currently) has restricted access to one or more repositories.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/ https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org http://bugs.python.org/ for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org http://bugs.python.org/? Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org http://bugs.python.org/ but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org http://bugs.python.org/ username is and then go check that manually.
There is CLAHub (https://www.clahub.com/ https://www.clahub.com/) but I don’t have any idea how good it is, I just know of it’s existence.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/ http://bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
This is the tool I used for the demo repo, it seemed to work ok as long as I ran it on Linux: https://github.com/frej/fast-export https://github.com/frej/fast-export It lets you annotate the git commits with the HG hash as a “git note”, though tool support for git notes doesn’t seem to be very good. Github doesn’t display them but they are available in the CLI if you run the right command to ask for the hg hash that a particular git commit hash came from.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Personally for most repositories I would just use the GitHub merge button.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
We don’t *actually* need to do much here. We could have a cronjob (or webhook based daemon or something) that used hg-git to do ``hg pull`` from GitHub onto hg.python.org (and probably a mirror on git.python.org too). That would A) allow read only tooling that is currently pointing at hg.python.org to continue to work unmodified and B) allow people to interact with our repos, in a read only fashion, without ever talking to GitHub. Couple that with the ability to still upload patches to the bug tracker and people can still contribute without ever personally sending a packet of data to Github. Of course it’d be nice to get the website itself pulling straight from Github (possible also using a web hook based daemon) though it could also either use hg-git, or just switch to Git. Either way should work.
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
Personally I feel like we should just have all of the branches live in the same repository. I don’t think there’s going to be much to gain by stripping out the other branches and I think that the downside of trying to work on 2+ repositories is a hefty price to pay. A fresh clone on the demo repo I setup has a .git of 156M.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
I think it will work best if fixes go into master. I find less problems with people writing patches/PRs against the wrong branch that way.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org http://bugs.python.org/ and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967 http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
I haven’t actually used it yet, but a friend has recently made https://pypi.python.org/pypi/towncrier https://pypi.python.org/pypi/towncrier which might be useful for this.
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
In case folks don’t know, github makes the PRs available as a remote HEAD that you can check out directly from a clone, might be useful for this.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
Git allows you to do it either way, by default it tracks the author and the committer separately so people will get credit, but if someone is just apply a diff then that obviously won’t happen by default (but people can still use the relevant options to make it happen).
Six, we will need to update our Buildbot fleet.
This gets us to the bare minimum needed to function.
Parity with hg.python.org http://hg.python.org/ ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org http://bugs.python.org/ that the PR exists. If people disagree then feel free to say so.
Github has a built in IRC bot for commits FWIW. I agree with the issue # bit.
Adding perks ========== Now we get to some added stuff that we never had on our own infrastructure. :)
We should wire up CI for all PRs. I don't know if we want to go with Travis, Codeship, or what CI provider, but we should definitely hook it up and fully utilize the resource. This could even include running doctest over the docs, making sure the reST markup is accurate, etc.
I like Travis a lot and I know the folks behind it, I’m sure they’d be happy to help.
Do we need to set up a web hook to trigger website rebuilds? We should at least have a mirror on Read the Docs that is triggered by web hook so that we have a backup of the documentation (if possible; not sure how custom our Sphinx setup is compared to what they require to work).
I’m sure Eric would be willing to help to make this happen.
We should try to get test coverage wired up as well per CI. I don't know if coveralls.io http://coveralls.io/ or some other provider is best, but we should see what is available and find out if we can use them to either get basic coverage or thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124 https://hg.python.org/devinabox/file/tip/README#l124 to see what thorough coverage entails, but it does require a checkout of coverage.py).
I prefer codecov, but it shouldn’t be too hard to do. I tried to get Python + C coverage checking in the demo with that, but I failed at making the C coverage work.
We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death scenes from the Holy Grail for inspiration since they deal with blocking you from doing something). It should handle specifying the commit message, what branches to commit to/cherry pick into, and a NEWS entry (if necessary). I don't know if it needs to do anything else as a requirement. It should probably implement a commit queue like Zuul or Homu (and both of those can be considered as the basis of the bot). Also gating commits on passing a test run probably would also be good.
I'm sure we will want to use some labels and milestones to track what PRs are for what versions, if they are blocked on something, etc.
---
Long email! :) I think that is my current brain dumped in email form. As I said at the beginning, I think we should focus on what is blocking the easiest repos first and then just keep knocking down blockers as we try to move over more repos. _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 5 January 2016 at 11:08, Donald Stufft
On Jan 4, 2016, at 7:42 PM, Brett Cannon
wrote: We should try to get test coverage wired up as well per CI. I don't know if coveralls.io or some other provider is best, but we should see what is available and find out if we can use them to either get basic coverage or thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124 to see what thorough coverage entails, but it does require a checkout of coverage.py).
I prefer codecov, but it shouldn’t be too hard to do. I tried to get Python + C coverage checking in the demo with that, but I failed at making the C coverage work.
Another posslble tool worth considering is diff_cover: https://pypi.python.org/pypi/diff_cover/ That uses git diff to find the lines affected by a patch and specifically looks up *those lines* in a coverage report, so it can ensure that any lines changed by a patch are covered by the regression test suite. It appears to be a neat way of guiding a code base towards more comprehensive test coverage. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Jan 4, 2016, at 10:45 PM, Nick Coghlan
wrote: On 5 January 2016 at 11:08, Donald Stufft
wrote: On Jan 4, 2016, at 7:42 PM, Brett Cannon
wrote: We should try to get test coverage wired up as well per CI. I don't know if coveralls.io or some other provider is best, but we should see what is available and find out if we can use them to either get basic coverage or thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124 to see what thorough coverage entails, but it does require a checkout of coverage.py).
I prefer codecov, but it shouldn’t be too hard to do. I tried to get Python + C coverage checking in the demo with that, but I failed at making the C coverage work.
Another posslble tool worth considering is diff_cover: https://pypi.python.org/pypi/diff_cover/
That uses git diff to find the lines affected by a patch and specifically looks up *those lines* in a coverage report, so it can ensure that any lines changed by a patch are covered by the regression test suite. It appears to be a neat way of guiding a code base towards more comprehensive test coverage.
FWIW codecov has that built in. ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft
There is CLAHub (https://www.clahub.com/) but I don’t have any idea how good
it is, I just know of it’s existence. CLAHub is apparently dying, see https://github.com/clahub/clahub/issues/111. Another option is https://cla-assistant.io/ which already has features that CLAHub doesn't, such as posting "You haven't signed the CLA" comments on review requests from new contributors. -- Nicolás Apologies for crappy quoting etc. I'm using Gmane to post this message.
On Jan 05, 2016, at 12:42 AM, Brett Cannon wrote:
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
Arthur: Each core dev converts four repos... Knight: Five repos Arthur: He who converts the repos four... Knight: Five repos Arthur: Five repos may hack in safety
There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I suspect cherry-picking between Python 2 and 3 branches won't be very smooth no matter what approach is chosen. Still, it's probably just as easy to keep everything in one repo.
Second, do we want all fixes to go into master and then cherry-pick into other branches
IME, yes, that usually works the best for the person doing the cherry picking. It also seems that all contributed PRs are always just made against master anyway.
We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death
You've clearly done your homework! Which probably involved watching MPatHG at least 10 times. :) Something else to consider. We've long talked about splitting out the stdlib to make it easier for the alternative implementations to import. If some or all of them also switch to git, we could do that pretty easily with git submodules. Cheers, -Barry
On Jan 4, 2016, at 9:42 PM, Barry Warsaw
wrote: Something else to consider. We've long talked about splitting out the stdlib to make it easier for the alternative implementations to import. If some or all of them also switch to git, we could do that pretty easily with git submodules.
They could also use a subtree merge - https://jrsmith3.github.io/merging-a-subdirectory-from-another-repo-via-git-... ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Something else to consider. We’ve long talked about splitting out the stdlib
to make it easier for the alternative implementations to import. If some or
all of them also switch to git, we could do that pretty easily with git
submodules.
Not to derail here, but wasn’t there a discussion (perhaps on python-ideas)
about slowly moving to a model where we distribute a barebones Python
“core”, allowing the standard modules to be updated and released on a more
frequent cycle? Would this be one small step towards such a model?
Nick
On Mon, Jan 4, 2016 at 9:43 PM Barry Warsaw
On Jan 05, 2016, at 12:42 AM, Brett Cannon wrote:
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
Arthur: Each core dev converts four repos... Knight: Five repos Arthur: He who converts the repos four... Knight: Five repos Arthur: Five repos may hack in safety
There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I suspect cherry-picking between Python 2 and 3 branches won't be very smooth no matter what approach is chosen. Still, it's probably just as easy to keep everything in one repo.
Second, do we want all fixes to go into master and then cherry-pick into other branches
IME, yes, that usually works the best for the person doing the cherry picking. It also seems that all contributed PRs are always just made against master anyway.
We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death
You've clearly done your homework! Which probably involved watching MPatHG at least 10 times. :)
Something else to consider. We've long talked about splitting out the stdlib to make it easier for the alternative implementations to import. If some or all of them also switch to git, we could do that pretty easily with git submodules.
Cheers, -Barry _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
On 5 January 2016 at 10:42, Brett Cannon
So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython. I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think we want at least one group to track CPython commit privileges and potentially a second to track CLA signatories. For collecting GitHub username info, I think it makes more sense to add a new User profile field in Roundup than it does to use a text file in the devguide: http://roundup.sourceforge.net/docs/customizing.html#tracker-schema That way whether or not someone has signed the CLA and whether or not they have commit privileges is directly associated with both their Roundup user ID and their GitHub one, but the latter is only linked if they choose to provide it. It also means that if we eventually have a Roundup hook submitting patches on behalf of people (perhaps triggered by a "Create PR" link on the patch display rather than implicitly), it can set "Author" on the PR correctly if the author has provided a GitHub username. We should also consider the question of who needs to be added to the admin group for the GitHub python organisation.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org? Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
The way kubernetes does this is that googlebot checks if the submitter has signed the CLA, and if they have it sets a green "cla: yes" flag on the PR: https://github.com/kubernetes/kubernetes/labels/cla%3A%20yes If they haven't, then it posts a message asking them to sign it and applies a red "cla: no" label: https://github.com/kubernetes/kubernetes/pull/19271#issuecomment-168836357 For us, I think the approach that makes the most sense depends on whether or not it's easy to query the Roundup user DB based on a custom field. If it's easy, then I think we should just have the bot query Roundup to ask "Is this GitHub user id linked to a user account that has signed the CLA?". If querying by custom field is a pain, then I think we should instead have a GitHub group to track CLA signatories and tweak Roundup to add someone to that group with their signatory status is updated. The bot would then check the derived GitHub group rather than querying Roundup directly.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Linear history is most useful for bisecting regressions, so I don't see a major need for it on any of the repos other than the main CPython one, while for at least the PEPs and the devguide I see a lot of value in enabling the low friction edit-PR-merge workflow for submitting small doc fixes (typos, etc).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
I like Donald's suggestion of setting up a webhook to ensure hg.python.org remains an up to date read-only Mercurial mirror. (Which suggests another important step: tweaking the configuration of those repos to block commits as part of the cutover of each repo) Something we may also want to consider is whether or not we might be able to use ReadTheDocs for building and hosting at least some of the ancillary repos (it would be nice to be able to use full Sphinx markup when writing PEPs, for example, and the devguide is already a Sphinx project).
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections.
Shallow clones are going to be more of a benefit there, and I agree with Donald that splitting the repos would be more trouble than it's worth.
We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
While I'd still like to see the tutorial and the HOWTO guides moved out to their own version independent repos in the long term, I don't see any reason to do that as part of the migration to a richer repository hosting environment - we can consider if and when we want to tackle it *after* the migration.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Master+cherry-pick makes sense to me. I have two concrete reasons for that, one personal and one professional: * the personal reason is that it means I can effectively ignore all but the most recent maintenance branch when contributing on my own time * the professional reason is that "please backport the fix for issue #ABC to version X.Y" requests become much easier to handle (even if X.Y is in security fix only mode), as it aligns with the normal upstream workflow rather than being an exceptional case
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
I'm generally a fan of loose coupling, and the main virtue I see for "in the repo" file-based approaches is that it means that everything you need to generate the release notes remains with the code and documentation - you're not relying on an external service being available. It also doesn't rule out the use of a tracker field later - such a field would just need to be converted into a file in the repo when preparing the patch.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
I think we'll still need the manual ACKS file to accommodate patches uploaded to Roundup (especially older ones), so I think the most useful thing to do here is to have a script that can check the ACKs file for missing names that appear in the GitHub contributor list.
We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death scenes from the Holy Grail for inspiration since they deal with blocking you from doing something).
I'm still a fan of "blue-no-green" :)
It should handle specifying the commit message, what branches to commit to/cherry pick into, and a NEWS entry (if necessary). I don't know if it needs to do anything else as a requirement. It should probably implement a commit queue like Zuul or Homu (and both of those can be considered as the basis of the bot). Also gating commits on passing a test run probably would also be good.
Something else worth considering is whether to have one bot or multiple. With the Kubernetes issue I linked above for example, you can see that the googlebot handles the CLA question, but there's a separate k8s-bot that prompts committers to move the issue along (and what appears to be a third party bot doing integration testing for Mesosphere). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Jan 5, 2016 at 5:18 AM, Nick Coghlan
On 5 January 2016 at 10:42, Brett Cannon
wrote: ...
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think we want at least one group to track CPython commit privileges and potentially a second to track CLA signatories.
For collecting GitHub username info, I think it makes more sense to add a new User profile field in Roundup than it does to use a text file in the devguide: http://roundup.sourceforge.net/docs/customizing.html#tracker-schema
Agreed.
That way whether or not someone has signed the CLA and whether or not they have commit privileges is directly associated with both their Roundup user ID and their GitHub one, but the latter is only linked if they choose to provide it.
It also means that if we eventually have a Roundup hook submitting patches on behalf of people (perhaps triggered by a "Create PR" link on the patch display rather than implicitly), it can set "Author" on the PR correctly if the author has provided a GitHub username.
We should also consider the question of who needs to be added to the admin group for the GitHub python organisation.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org? Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
The way kubernetes does this is that googlebot checks if the submitter has signed the CLA, and if they have it sets a green "cla: yes" flag on the PR: https://github.com/kubernetes/kubernetes/labels/cla%3A%20yes
If they haven't, then it posts a message asking them to sign it and applies a red "cla: no" label: https://github.com/kubernetes/kubernetes/pull/19271#issuecomment-168836357
For us, I think the approach that makes the most sense depends on whether or not it's easy to query the Roundup user DB based on a custom field. If it's easy, then I think we should just have the bot query Roundup to ask "Is this GitHub user id linked to a user account that has signed the CLA?".
It should be easy. Best Regards, Ezio Melotti
If querying by custom field is a pain, then I think we should instead have a GitHub group to track CLA signatories and tweak Roundup to add someone to that group with their signatory status is updated. The bot would then check the derived GitHub group rather than querying Roundup directly.
On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon
So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
On top of this, there is also the test repo (https://hg.python.org/test) and all the tracker repos (https://hg.python.org/tracker/). I think it would be useful to port the former since it will provide a place for devs to try things out and experiment (a new test repo could also be created though). It would be nice to port the tracker repos too and be consistent with the others, but it's not a priority. When we switched to HG they kept being on SVN until I ported them, so I guess the same thing can be done (unless R. David or Martin prefer to stick to HG).
I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think the current list of core-devs should be converted to a group and given access to the same repos they have access to now (i.e. cpython/devguide/peps and possibly others). Then additional repo-specific groups can be created in case we want to let specific contributors work on peps or the devguide.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org?
This can be done. We can add a "GitHub" username field to Roundup users so that we can link the two.
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice. It would be better if users didn't need to create and use two separate accounts.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
As I mentioned on python-committers, we already have https://hg.python.org/lookup/ . This is currently used to map SVN->HG (e.g. https://hg.python.org/lookup/r12345 ), and should be extended to handle cs ids too. The b.p.o linkifier can just convert all revision numbers and cs ids to a https://hg.python.org/lookup/ link and let the lookup page figure out where to redirect the user.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
I think we should make hg.python.org read-only but keep it around and in sync with the GitHub repo (either via cronjobs or hooks). This will allow people to contribute using HG in the same way that the current GitHub clone allows people to contribute using git. It will also avoid breaking all the tools that currently use hg.python.org (and buys us more time to port them if/when needed).
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I think we should keep 2/3 together. We could split the stdlib from the rest, but that's a separate issue.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Master first and cherry-picking for older branches sounds good to me, but I don't know if switching model will have any implications, especially while going through the history or using tools like bisect.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
I would like to see a complete list of steps from starting to work on an issue to having it in the repo, at least to understand the new workflow. This doesn't have to include all the specific commands, but at least the basic steps (e.g. after I made a patch to I commit it and send a pull request to the main repo, or do I push it to my GitHub clone and push a button to send the PR? Do I need to create a branch before I start working on an issue?
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
We could keep updating for regular patches with no related PR and add a note about all the other GIT contributors (possibly with a git command that lists all authors). Later on we might decide to have a script that automatically adds all the GIT contributors automatically.
Six, we will need to update our Buildbot fleet.
If we keep hg.p.o around and updated, we might not have to do this now (even though now is better than never).
This gets us to the bare minimum needed to function.
Parity with hg.python.org ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org that the PR exists. If people disagree then feel free to say so.
FWIW I started adding notes to https://wiki.python.org/moin/TrackerDevelopmentPlanning to track everything that needs to be done on the Roundup side. If you prefer I can later move this to the new PEP, but for now I'm using it to keep track of all the things that come up in the various threads. Best Regards, Ezio Melotti
Adding perks ========== ...
On Jan 5, 2016, at 12:53 AM, Ezio Melotti
wrote: Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
Can we allow (not mandate) login to b.p.o with a GitHub account? That should solve #1 and should sort of solve #3 as well. Ideally it’d allow you to associate a Github account with an existing b.p.o account (and doing so should also automatically create an association that X on b.p.o is Y on Github). ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On 05.01.2016 06:53, Ezio Melotti wrote:
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
Given that we want to make it possible to move away from Github without too much fuzz, wouldn't it be better to have the PR discussions on b.p.o and Rietvield ? If we start using Github for this, we'd lose that part of the review history when moving away. Moving from the SF issue tracker to b.p.o was a major piece of work (mostly done by Martin von Löwis IIRC) and it's not clear how we could retrofit those discussions into the b.p.o issue discussions. Perhaps we could gateway the emails that Github sends for PR discussions back into b.p.o in some way (the emails contain the PR number, repo name and Github account name of the sender in the headers). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 05 2016)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Tue, 5 Jan 2016 at 03:22 M.-A. Lemburg
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what
On 05.01.2016 06:53, Ezio Melotti wrote: their
bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
Given that we want to make it possible to move away from Github without too much fuzz, wouldn't it be better to have the PR discussions on b.p.o and Rietvield ?
One of the motivating factors of this move is to get off of our fork of Rietveld, so that's not feasible.
If we start using Github for this, we'd lose that part of the review history when moving away.
GitHub's API allows for exporting all of the data.
Moving from the SF issue tracker to b.p.o was a major piece of work (mostly done by Martin von Löwis IIRC) and it's not clear how we could retrofit those discussions into the b.p.o issue discussions.
Perhaps we could gateway the emails that Github sends for PR discussions back into b.p.o in some way (the emails contain the PR number, repo name and Github account name of the sender in the headers).
I believe GitLab has a GitHub -> GitLab migration tool that keeps PR history, so this isn't quite the issue that it initially appears to be. If people are that worried, we could do a daily dump of the data. But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
On 05.01.2016 18:50, Brett Cannon wrote:
On Tue, 5 Jan 2016 at 03:22 M.-A. Lemburg
wrote: Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what
On 05.01.2016 06:53, Ezio Melotti wrote: their
bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
Given that we want to make it possible to move away from Github without too much fuzz, wouldn't it be better to have the PR discussions on b.p.o and Rietvield ?
One of the motivating factors of this move is to get off of our fork of Rietveld, so that's not feasible.
If we start using Github for this, we'd lose that part of the review history when moving away.
GitHub's API allows for exporting all of the data.
Moving from the SF issue tracker to b.p.o was a major piece of work (mostly done by Martin von Löwis IIRC) and it's not clear how we could retrofit those discussions into the b.p.o issue discussions.
Perhaps we could gateway the emails that Github sends for PR discussions back into b.p.o in some way (the emails contain the PR number, repo name and Github account name of the sender in the headers).
I believe GitLab has a GitHub -> GitLab migration tool that keeps PR history, so this isn't quite the issue that it initially appears to be.
If that's the case, then it's fine.
If people are that worried, we could do a daily dump of the data.
This would be a good idea in general. Backups are always good to have :-)
But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
True. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Jan 05 2016)
Python Projects, Coaching and Consulting ... http://www.egenix.com/ Python Database Interfaces ... http://products.egenix.com/ Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
On Tue, 05 Jan 2016 17:50:53 +0000, Brett Cannon
If people are that worried, we could do a daily dump of the data. But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
We can fairly easily have all github-originated comments not trigger an email to the nosy list. We'd just need to add a check for the originating email address to the nosy reaction detector. Even nicer would be to make it a per-user setting (I'd rather get the emails from b.p.o. and disable them on github, myself), but that would be a bit more work. It would be nice to have *all* of the discussion in one place. --David
On Tue, 5 Jan 2016 at 10:48 R. David Murray
On Tue, 05 Jan 2016 17:50:53 +0000, Brett Cannon
wrote: If people are that worried, we could do a daily dump of the data. But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
We can fairly easily have all github-originated comments not trigger an email to the nosy list. We'd just need to add a check for the originating email address to the nosy reaction detector. Even nicer would be to make it a per-user setting (I'd rather get the emails from b.p.o. and disable them on github, myself), but that would be a bit more work.
It would be nice to have *all* of the discussion in one place.
If someone wants to put the work in to make that happen then that's fine by me, but I do consider this a nicety and not a blocker. We haven't had code review comments go into b.p.o on Rietveld either so this isn't that far off from what we have now, especially if we block the transition on adding a link back to the GitHub PR through a web hook.
On Tue, Jan 5, 2016 at 9:18 PM, Brett Cannon
On Tue, 5 Jan 2016 at 10:48 R. David Murray
wrote: On Tue, 05 Jan 2016 17:50:53 +0000, Brett Cannon
wrote: If people are that worried, we could do a daily dump of the data. But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
We can fairly easily have all github-originated comments not trigger an email to the nosy list. We'd just need to add a check for the originating email address to the nosy reaction detector. Even nicer would be to make it a per-user setting (I'd rather get the emails from b.p.o. and disable them on github, myself), but that would be a bit more work.
It would be nice to have *all* of the discussion in one place.
If someone wants to put the work in to make that happen then that's fine by me, but I do consider this a nicety and not a blocker. We haven't had code review comments go into b.p.o on Rietveld either so this isn't that far off from what we have now, especially if we block the transition on adding a link back to the GitHub PR through a web hook.
I agree it's not a blocker, but a better integration between Rietveld and b.p.o was often requested (I tried -- and failed -- to tackle the issue a couple of times). What I want to avoid is having people on github missing b.p.o messages, and people on b.p.o missing PRs/reviews on github. (This was already happening when people were discussing the patch on Rietveld, and others were only looking at b.p.o and/or missing the reviews, both because the email from rietveld ended up in the spam folder, and because there was no way to know from b.p.o if someone posted a review.) I think it's ok to have discussions on b.p.o and reviews on github (similarly to what we currently do with Rietveld), as long as the everyone gets notified. Discussions that are not strictly related to the code/review shouldn't happen on github. As for the notifications, I think the best option is: * when a PR is posted, it's also automatically added to the issue (to the list of PRs, similar to the list of patches; generates an email too); * when a review is posted, a new message with a link is added to the issue (this is like a regular message, and generates an email too); * when someone posts a PR/review on github, they will also be added automatically on the nosy list of the issue on b.p.o (so they will receive b.p.o messages); * when a PR/review is posted, people in the issue nosy on b.p.o are NOT added to the PR nosy list (they already get new PRs/reviews notifications from b.p.o). This means that: * when a PR is posted: * the PR author will get a mail from b.p.o confirming that the PR has been added to the issue if they have a linked account (possibly on top of any email github might send for creating a PR); * people in the issue nosy on b.p.o will get an email that lets them know a PR has been added; * when a review is posted: * the PR author and the reviewers will get two emails (one from github and one from b.p.o if they have a linked account) or one (from github, if they don't); * people in the issue nosy will only get one email (from b.p.o, with a link to the review); * when a message is posted: * the PR author and reviewers will get one email (from b.p.o if they have a linked account) or zero (if they don't); * people in the issue nosy will get one email (from b.p.o); The "problematic" cases are: * the PR author and the reviewers will receive two emails for reviews if they have a linked account; * the PR author and the reviewers will receive zero emails for b.p.o messages if they don't have a linked account; The former can be solved if we add a way to disable review emails from b.p.o or if github provides one already (or using email filters). The latter can't be solved unless they link accounts (I don't think b.p.o messages should go to github). FWIW I would personally prefer to get all emails from b.p.o, either ignoring/filtering duplicated review emails from github, or disabling them from github if possible. If you agree, this is what needs to be done: 1) automatically add PRs to b.p.o issues; 2) automatically add a message on b.p.o when a review is posted on github; 3) add a "github username" field to b.p.o users to link accounts; 4) automatically add the PR author (during step 1) and reviewers (during step 2) to the issue nosy list on b.p.o; 5) add an option to disable review emails (optional); I would consider all these points except the 5th as blockers. Best Regards, Ezio Melotti
On Tue, 5 Jan 2016 at 23:46 Ezio Melotti
On Tue, Jan 5, 2016 at 9:18 PM, Brett Cannon
wrote: On Tue, 5 Jan 2016 at 10:48 R. David Murray
wrote:
On Tue, 05 Jan 2016 17:50:53 +0000, Brett Cannon
wrote:
If people are that worried, we could do a daily dump of the data. But unless we can have all GitHub-related comments to an issue not trigger an email update I don't think that's feasible as it would mean two emails for every PR comment; one from GitHub and one from b.p.o.
We can fairly easily have all github-originated comments not trigger an email to the nosy list. We'd just need to add a check for the originating email address to the nosy reaction detector. Even nicer would be to make it a per-user setting (I'd rather get the emails from b.p.o. and disable them on github, myself), but that would be a bit more work.
It would be nice to have *all* of the discussion in one place.
If someone wants to put the work in to make that happen then that's fine by me, but I do consider this a nicety and not a blocker. We haven't had code review comments go into b.p.o on Rietveld either so this isn't that far off from what we have now, especially if we block the transition on adding a link back to the GitHub PR through a web hook.
I agree it's not a blocker, but a better integration between Rietveld and b.p.o was often requested (I tried -- and failed -- to tackle the issue a couple of times). What I want to avoid is having people on github missing b.p.o messages, and people on b.p.o missing PRs/reviews on github. (This was already happening when people were discussing the patch on Rietveld, and others were only looking at b.p.o and/or missing the reviews, both because the email from rietveld ended up in the spam folder, and because there was no way to know from b.p.o if someone posted a review.)
I think it's ok to have discussions on b.p.o and reviews on github (similarly to what we currently do with Rietveld), as long as the everyone gets notified. Discussions that are not strictly related to the code/review shouldn't happen on github.
As for the notifications, I think the best option is: * when a PR is posted, it's also automatically added to the issue (to the list of PRs, similar to the list of patches; generates an email too); * when a review is posted, a new message with a link is added to the issue (this is like a regular message, and generates an email too); * when someone posts a PR/review on github, they will also be added automatically on the nosy list of the issue on b.p.o (so they will receive b.p.o messages); * when a PR/review is posted, people in the issue nosy on b.p.o are NOT added to the PR nosy list (they already get new PRs/reviews notifications from b.p.o).
This means that: * when a PR is posted: * the PR author will get a mail from b.p.o confirming that the PR has been added to the issue if they have a linked account (possibly on top of any email github might send for creating a PR); * people in the issue nosy on b.p.o will get an email that lets them know a PR has been added; * when a review is posted: * the PR author and the reviewers will get two emails (one from github and one from b.p.o if they have a linked account) or one (from github, if they don't); * people in the issue nosy will only get one email (from b.p.o, with a link to the review); * when a message is posted: * the PR author and reviewers will get one email (from b.p.o if they have a linked account) or zero (if they don't); * people in the issue nosy will get one email (from b.p.o);
The "problematic" cases are: * the PR author and the reviewers will receive two emails for reviews if they have a linked account; * the PR author and the reviewers will receive zero emails for b.p.o messages if they don't have a linked account;
The former can be solved if we add a way to disable review emails from b.p.o or if github provides one already (or using email filters). The latter can't be solved unless they link accounts (I don't think b.p.o messages should go to github).
FWIW I would personally prefer to get all emails from b.p.o, either ignoring/filtering duplicated review emails from github, or disabling them from github if possible.
If you agree, this is what needs to be done: 1) automatically add PRs to b.p.o issues;
This is a blocker.
2) automatically add a message on b.p.o when a review is posted on github;
With the way GitHub does reviews, this won't make sense. Each comment on a PR is essentially its own review (as Eric complained about because that means each comment is treated as a separate thing and thus generates its own email). It isn't like with Rietveld where there is a "Review + Email" step to send draft reviews and hence a review is a group of comments.
3) add a "github username" field to b.p.o users to link accounts;
That's a blocker for CLA enforcement.
4) automatically add the PR author (during step 1) and reviewers (during step 2) to the issue nosy list on b.p.o;
I view this as a great nicety but not a blocker.
5) add an option to disable review emails (optional);
This will have to be discussed in connection with the emails per PR comment in case there is a misunderstanding of what a review on GitHub is.
I would consider all these points except the 5th as blockers.
Obviously I don't think they all are, but definitely some. -Brett
Best Regards, Ezio Melotti
On Wed, Jan 6, 2016 at 8:18 PM, Brett Cannon
On Tue, 5 Jan 2016 at 23:46 Ezio Melotti
wrote: ... If you agree, this is what needs to be done: 1) automatically add PRs to b.p.o issues;
This is a blocker.
2) automatically add a message on b.p.o when a review is posted on github;
With the way GitHub does reviews, this won't make sense. Each comment on a PR is essentially its own review (as Eric complained about because that means each comment is treated as a separate thing and thus generates its own email). It isn't like with Rietveld where there is a "Review + Email" step to send draft reviews and hence a review is a group of comments.
Good to know. I can think of two alternative solutions: 1) have a "number of comments" and "date/time of the last comment" next to the PR, and update it whenever a new comment is posted; 2) check periodically for new comments, aggregate them somehow, and post a "X new comments have been added to PR Y." message;
3) add a "github username" field to b.p.o users to link accounts;
That's a blocker for CLA enforcement.
4) automatically add the PR author (during step 1) and reviewers (during step 2) to the issue nosy list on b.p.o;
I view this as a great nicety but not a blocker.
OK, I considered this a blocker because otherwise PR authors and reviewers might miss b.p.o comments, but we can let them know manually until it's implemented. Best Regards, Ezio Melotti
5) add an option to disable review emails (optional);
This will have to be discussed in connection with the emails per PR comment in case there is a misunderstanding of what a review on GitHub is.
I would consider all these points except the 5th as blockers.
Obviously I don't think they all are, but definitely some.
-Brett
Best Regards, Ezio Melotti
On Wed, 6 Jan 2016 at 23:56 Ezio Melotti
On Wed, Jan 6, 2016 at 8:18 PM, Brett Cannon
wrote: On Tue, 5 Jan 2016 at 23:46 Ezio Melotti
wrote: ... If you agree, this is what needs to be done: 1) automatically add PRs to b.p.o issues;
This is a blocker.
2) automatically add a message on b.p.o when a review is posted on
github;
With the way GitHub does reviews, this won't make sense. Each comment on
a
PR is essentially its own review (as Eric complained about because that means each comment is treated as a separate thing and thus generates its own email). It isn't like with Rietveld where there is a "Review + Email" step to send draft reviews and hence a review is a group of comments.
Good to know. I can think of two alternative solutions: 1) have a "number of comments" and "date/time of the last comment" next to the PR, and update it whenever a new comment is posted; 2) check periodically for new comments, aggregate them somehow, and post a "X new comments have been added to PR Y." message;
Either of those ideas work for me. Would option #2 be done as a comment and thus send out what amounts to a summary email of PR review comments? As long as we do that **only** when there have been changes and not blindly doing it every day (e.g., no "0 comments since yesterday" email), that would be acceptable to me in terms of email volume. Do the other people who wanted to follow PRs on b.p.o like that level of frequency?
3) add a "github username" field to b.p.o users to link accounts;
That's a blocker for CLA enforcement.
4) automatically add the PR author (during step 1) and reviewers (during step 2) to the issue nosy list on b.p.o;
I view this as a great nicety but not a blocker.
OK, I considered this a blocker because otherwise PR authors and reviewers might miss b.p.o comments, but we can let them know manually until it's implemented.
Yep, exactly. I want to block on key stuff that would make GitHub worse than what we have now, but not on stuff that makes it better. This is obviously not to say that I don't want to see all of these great improvements happen, but I do want to get over to the new workflow sooner rather than later as I suspect contributors will find it much easier to work with. -Brett
Best Regards, Ezio Melotti
5) add an option to disable review emails (optional);
This will have to be discussed in connection with the emails per PR comment in case there is a misunderstanding of what a review on GitHub is.
I would consider all these points except the 5th as blockers.
Obviously I don't think they all are, but definitely some.
-Brett
Best Regards, Ezio Melotti
On Thu, Jan 7, 2016 at 7:21 PM, Brett Cannon
On Wed, 6 Jan 2016 at 23:56 Ezio Melotti
wrote: On Wed, Jan 6, 2016 at 8:18 PM, Brett Cannon
wrote: On Tue, 5 Jan 2016 at 23:46 Ezio Melotti
wrote: ... If you agree, this is what needs to be done: 1) automatically add PRs to b.p.o issues;
This is a blocker.
2) automatically add a message on b.p.o when a review is posted on github;
With the way GitHub does reviews, this won't make sense. Each comment on a PR is essentially its own review (as Eric complained about because that means each comment is treated as a separate thing and thus generates its own email). It isn't like with Rietveld where there is a "Review + Email" step to send draft reviews and hence a review is a group of comments.
Good to know. I can think of two alternative solutions: 1) have a "number of comments" and "date/time of the last comment" next to the PR, and update it whenever a new comment is posted; 2) check periodically for new comments, aggregate them somehow, and post a "X new comments have been added to PR Y." message;
Either of those ideas work for me. Would option #2 be done as a comment and thus send out what amounts to a summary email of PR review comments? As long as we do that **only** when there have been changes and not blindly doing it every day (e.g., no "0 comments since yesterday" email), that would be acceptable to me in terms of email volume. Do the other people who wanted to follow PRs on b.p.o like that level of frequency?
The goal is to generate at least 1 message/email if a review (possibly comprising several comments) is posted. If there are no new comments, nothing is posted, but if there are new comments, a new message will be posted at some point. We just need to find a compromise between delay and number of messages (and that's something we can figure out later with a bit of trial and error). Checking daily might result in hours of delay, but no more than one daily message. Checking hourly has less delay, but it might result in more messages. We can also try to do something smarter by checking e.g. every 15 minutes and posting the message only if no new messages have been added in the last 15 minutes (so the reviewer has likely finished commenting).
3) add a "github username" field to b.p.o users to link accounts;
That's a blocker for CLA enforcement.
4) automatically add the PR author (during step 1) and reviewers (during step 2) to the issue nosy list on b.p.o;
I view this as a great nicety but not a blocker.
OK, I considered this a blocker because otherwise PR authors and reviewers might miss b.p.o comments, but we can let them know manually until it's implemented.
Yep, exactly. I want to block on key stuff that would make GitHub worse than what we have now, but not on stuff that makes it better.
This is a good way to see it. Best Regards, Ezio Melotti
This is obviously not to say that I don't want to see all of these great improvements happen, but I do want to get over to the new workflow sooner rather than later as I suspect contributors will find it much easier to work with.
-Brett
Best Regards, Ezio Melotti
5) add an option to disable review emails (optional);
This will have to be discussed in connection with the emails per PR comment in case there is a misunderstanding of what a review on GitHub is.
I would consider all these points except the 5th as blockers.
Obviously I don't think they all are, but definitely some.
-Brett
Best Regards, Ezio Melotti
On Thu, Jan 7, 2016 at 12:40 PM, Ezio Melotti
We can also try to do something smarter by checking e.g. every 15 minutes and posting the message only if no new messages have been added in the last 15 minutes (so the reviewer has likely finished commenting).
I like this plan, though 15 minutes might be a bit short for the cooldown period; half an hour might be a bit better. On the other hand, it is possible to write out all your review comments without hitting the 'comment' button on any of them, then scroll through and click comment on each of them, in which case the cooldown could be even shorter than 15 minutes. Alternately, what about checking for a marker in a top-level (not a line note) comment, which signals "I finished a review, aggregate it and post it to b.p.o"? -- Zach
On Thu, 7 Jan 2016 at 11:05 Zachary Ware
On Thu, Jan 7, 2016 at 12:40 PM, Ezio Melotti
wrote: We can also try to do something smarter by checking e.g. every 15 minutes and posting the message only if no new messages have been added in the last 15 minutes (so the reviewer has likely finished commenting).
I like this plan, though 15 minutes might be a bit short for the cooldown period; half an hour might be a bit better. On the other hand, it is possible to write out all your review comments without hitting the 'comment' button on any of them, then scroll through and click comment on each of them, in which case the cooldown could be even shorter than 15 minutes.
Alternately, what about checking for a marker in a top-level (not a line note) comment, which signals "I finished a review, aggregate it and post it to b.p.o"?
The problem is simply forgetting to leave that marker since it isn't required for someone's review comments to be seen in GitHub.
On Thu, Jan 7, 2016 at 1:19 PM, Brett Cannon
On Thu, 7 Jan 2016 at 11:05 Zachary Ware
wrote: On Thu, Jan 7, 2016 at 12:40 PM, Ezio Melotti
wrote: We can also try to do something smarter by checking e.g. every 15 minutes and posting the message only if no new messages have been added in the last 15 minutes (so the reviewer has likely finished commenting).
I like this plan, though 15 minutes might be a bit short for the cooldown period; half an hour might be a bit better. On the other hand, it is possible to write out all your review comments without hitting the 'comment' button on any of them, then scroll through and click comment on each of them, in which case the cooldown could be even shorter than 15 minutes.
Alternately, what about checking for a marker in a top-level (not a line note) comment, which signals "I finished a review, aggregate it and post it to b.p.o"?
The problem is simply forgetting to leave that marker since it isn't required for someone's review comments to be seen in GitHub.
In that case, what about using a marker in combination with a longer cooldown automatic aggregator? If you remember the marker, it posts immediately and is more likely to do what you want; if you forget the marker, it happens eventually anyway. Or just forget the marker entirely, and treat any top-level comment on the PR as "there's something that should be posted to b.p.o now". That method fits better with what we're used to with Rietveld. -- Zach
On Thu, Jan 7, 2016 at 8:40 PM, Ezio Melotti
The goal is to generate at least 1 message/email if a review (possibly comprising several comments) is posted. If there are no new comments, nothing is posted, but if there are new comments, a new message will be posted at some point. We just need to find a compromise between delay and number of messages (and that's something we can figure out later with a bit of trial and error). Checking daily might result in hours of delay, but no more than one daily message. Checking hourly has less delay, but it might result in more messages. We can also try to do something smarter by checking e.g. every 15 minutes and posting the message only if no new messages have been added in the last 15 minutes (so the reviewer has likely finished commenting).
I think this still will create too much noise. I'd prefer not to see comments like "this needs to be tested", "needs versionadded", "please don't change function signature" etc. in issues. I like following Windows and IDLE issues, but I'm not really interested seeing review comments about them, for example. Wouldn't a new pull request field in the issue detail page be enough to link pull requests? Django uses a similar solution to this: https://code.djangoproject.com/ticket/25995 (see "Pull Requests: 5928 build:success") We could show total number of comments too.
In a message of Tue, 05 Jan 2016 12:22:03 +0100, "M.-A. Lemburg" writes:
Given that we want to make it possible to move away from Github without too much fuzz, wouldn't it be better to have the PR discussions on b.p.o and Rietvield ?
+1
If we start using Github for this, we'd lose that part of the review history when moving away.
Moving from the SF issue tracker to b.p.o was a major piece of work (mostly done by Martin von Löwis IIRC) and it's not clear how we could retrofit those discussions into the b.p.o issue discussions.
You recall very correctly. Laura
On Mon, 4 Jan 2016 at 21:54 Ezio Melotti
On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon
wrote: So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
On top of this, there is also the test repo (https://hg.python.org/test) and all the tracker repos (https://hg.python.org/tracker/). I think it would be useful to port the former since it will provide a place for devs to try things out and experiment (a new test repo could also be created though). It would be nice to port the tracker repos too and be consistent with the others, but it's not a priority. When we switched to HG they kept being on SVN until I ported them, so I guess the same thing can be done (unless R. David or Martin prefer to stick to HG).
I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think the current list of core-devs should be converted to a group and given access to the same repos they have access to now (i.e. cpython/devguide/peps and possibly others). Then additional repo-specific groups can be created in case we want to let specific contributors work on peps or the devguide.
This seems to be the general consensus, so we will create a python-dev team under the python org and add the core devs there.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org?
This can be done. We can add a "GitHub" username field to Roundup users so that we can link the two.
OK, so it sounds like we will stick with our current CLA signing flow and write our own CLA bot that will query Roundup as to whether someone has signed the CLA or not and then throw up a banner signalling if someone has (not) signed and an appropriate link to the CLA. That will require some Roundup work and the creation of the bot. I should also mention, any bot creations we do should abstract out the code review tool so that when we change providers again in the future it will be more straight-forward to just update some select APIs rather than rewrite every bot we create.
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
If we can add GitHub as a login/creation option for b.p.o accounts then that solves that. But I'm willing to bet a majority of people will already have a GitHub account and we have always required the b.p.o account so #1 is the going to be the common case.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
As I mentioned on python-committers, we already have https://hg.python.org/lookup/ . This is currently used to map SVN->HG (e.g. https://hg.python.org/lookup/r12345 ), and should be extended to handle cs ids too. The b.p.o linkifier can just convert all revision numbers and cs ids to a https://hg.python.org/lookup/ link and let the lookup page figure out where to redirect the user.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
I think we should make hg.python.org read-only but keep it around and in sync with the GitHub repo (either via cronjobs or hooks). This will allow people to contribute using HG in the same way that the current GitHub clone allows people to contribute using git. It will also avoid breaking all the tools that currently use hg.python.org (and buys us more time to port them if/when needed).
That's easy to say, but someone also has to maintain hg.python.org then and we are doing this move partially to try and cut down on the amount of custom infrastructure that we maintain. If people are that worried about others being so adverse to using GitHub that they won't even do an anonymous clone from their servers then we can get a Bitbucket or GitLab clone set up, but I would rather try and cut out our repo hosting services if possible (who knows, maybe we can even finally retire svn.python.org thanks to shallow clones or something).
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I think we should keep 2/3 together. We could split the stdlib from the rest, but that's a separate issue.
This seems to be the general consensus, so we will plan to keep cpython as a single repo.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Master first and cherry-picking for older branches sounds good to me, but I don't know if switching model will have any implications, especially while going through the history or using tools like bisect.
This seems to be the general consensus, so we will plan to cherry pick commits into older branches.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
I would like to see a complete list of steps from starting to work on an issue to having it in the repo, at least to understand the new workflow. This doesn't have to include all the specific commands, but at least the basic steps (e.g. after I made a patch to I commit it and send a pull request to the main repo, or do I push it to my GitHub clone and push a button to send the PR? Do I need to create a branch before I start working on an issue?
There will be a step-by-step guide in the devguide to answer all of this before we make any switch.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
We could keep updating for regular patches with no related PR and add a note about all the other GIT contributors (possibly with a git command that lists all authors). Later on we might decide to have a script that automatically adds all the GIT contributors automatically.
This seems to be the general consensus, so we will keep Misc/ACKS around and have a tool that updates it based on git PR commits at release-time.
Six, we will need to update our Buildbot fleet.
If we keep hg.p.o around and updated, we might not have to do this now (even though now is better than never).
This gets us to the bare minimum needed to function.
Parity with hg.python.org ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org that the PR exists. If people disagree then feel free to say so.
FWIW I started adding notes to https://wiki.python.org/moin/TrackerDevelopmentPlanning to track everything that needs to be done on the Roundup side. If you prefer I can later move this to the new PEP, but for now I'm using it to keep track of all the things that come up in the various threads.
Nope, the wiki is fine for that sort of thing. -Brett
On Jan 5, 2016, at 1:03 PM, Brett Cannon
wrote: On Mon, 4 Jan 2016 at 21:54 Ezio Melotti
mailto:ezio.melotti@gmail.com> wrote: On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon mailto:brett@python.org> wrote: So consider this the starting discussion of the PEP that will be the hg.python.org http://hg.python.org/ -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
On top of this, there is also the test repo (https://hg.python.org/test https://hg.python.org/test) and all the tracker repos (https://hg.python.org/tracker/ https://hg.python.org/tracker/). I think it would be useful to port the former since it will provide a place for devs to try things out and experiment (a new test repo could also be created though). It would be nice to port the tracker repos too and be consistent with the others, but it's not a priority. When we switched to HG they kept being on SVN until I ported them, so I guess the same thing can be done (unless R. David or Martin prefer to stick to HG).
I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think the current list of core-devs should be converted to a group and given access to the same repos they have access to now (i.e. cpython/devguide/peps and possibly others). Then additional repo-specific groups can be created in case we want to let specific contributors work on peps or the devguide.
This seems to be the general consensus, so we will create a python-dev team under the python org and add the core devs there.
Just to expand on this more, GitHub has recently updated their permission model more to also allow “outside contributors” to be added to a single repository without adding them to a group (Not sure if this would be useful for us or not). I imagine the python-dev team would have “Write Access” (See https://help.github.com/articles/repository-permission-levels-for-an-organiz... https://help.github.com/articles/repository-permission-levels-for-an-organiz...) unless we think it needs admin access.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/ https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org http://bugs.python.org/ for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org http://bugs.python.org/?
This can be done. We can add a "GitHub" username field to Roundup users so that we can link the two.
OK, so it sounds like we will stick with our current CLA signing flow and write our own CLA bot that will query Roundup as to whether someone has signed the CLA or not and then throw up a banner signalling if someone has (not) signed and an appropriate link to the CLA. That will require some Roundup work and the creation of the bot.
We can set a commit status that will show red if the user hasn’t signed the CLA (just like if Travis tests failed or so). No need to use a banner or anything.
I should also mention, any bot creations we do should abstract out the code review tool so that when we change providers again in the future it will be more straight-forward to just update some select APIs rather than rewrite every bot we create.
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org http://bugs.python.org/ but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org http://bugs.python.org/ username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
If we can add GitHub as a login/creation option for b.p.o accounts then that solves that. But I'm willing to bet a majority of people will already have a GitHub account and we have always required the b.p.o account so #1 is the going to be the common case.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/ http://bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
As I mentioned on python-committers, we already have https://hg.python.org/lookup/ https://hg.python.org/lookup/ . This is currently used to map SVN->HG (e.g. https://hg.python.org/lookup/r12345 https://hg.python.org/lookup/r12345 ), and should be extended to handle cs ids too. The b.p.o linkifier can just convert all revision numbers and cs ids to a https://hg.python.org/lookup/ https://hg.python.org/lookup/ link and let the lookup page figure out where to redirect the user.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
I think we should make hg.python.org http://hg.python.org/ read-only but keep it around and in sync with the GitHub repo (either via cronjobs or hooks). This will allow people to contribute using HG in the same way that the current GitHub clone allows people to contribute using git. It will also avoid breaking all the tools that currently use hg.python.org http://hg.python.org/ (and buys us more time to port them if/when needed).
That's easy to say, but someone also has to maintain hg.python.org http://hg.python.org/ then and we are doing this move partially to try and cut down on the amount of custom infrastructure that we maintain. If people are that worried about others being so adverse to using GitHub that they won't even do an anonymous clone from their servers then we can get a Bitbucket or GitLab clone set up, but I would rather try and cut out our repo hosting services if possible (who knows, maybe we can even finally retire svn.python.org http://svn.python.org/ thanks to shallow clones or something).
I think a mirror is a different level of involvement than running the main repository. We won’t have to worry about backups, if the server goes down people can still contribute, etc. Most likely it will also be able to be resized to a smaller server as well (but maybe not). I don’t think any of the infra team would be upset at one less server to manage though, so I’m more than happy either way.
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I think we should keep 2/3 together. We could split the stdlib from the rest, but that's a separate issue.
This seems to be the general consensus, so we will plan to keep cpython as a single repo.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Master first and cherry-picking for older branches sounds good to me, but I don't know if switching model will have any implications, especially while going through the history or using tools like bisect.
This seems to be the general consensus, so we will plan to cherry pick commits into older branches.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org http://bugs.python.org/ and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967 http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
I would like to see a complete list of steps from starting to work on an issue to having it in the repo, at least to understand the new workflow. This doesn't have to include all the specific commands, but at least the basic steps (e.g. after I made a patch to I commit it and send a pull request to the main repo, or do I push it to my GitHub clone and push a button to send the PR? Do I need to create a branch before I start working on an issue?
There will be a step-by-step guide in the devguide to answer all of this before we make any switch.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
We could keep updating for regular patches with no related PR and add a note about all the other GIT contributors (possibly with a git command that lists all authors). Later on we might decide to have a script that automatically adds all the GIT contributors automatically.
This seems to be the general consensus, so we will keep Misc/ACKS around and have a tool that updates it based on git PR commits at release-time.
Six, we will need to update our Buildbot fleet.
If we keep hg.p.o around and updated, we might not have to do this now (even though now is better than never).
This gets us to the bare minimum needed to function.
Parity with hg.python.org http://hg.python.org/ ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org http://bugs.python.org/ that the PR exists. If people disagree then feel free to say so.
FWIW I started adding notes to https://wiki.python.org/moin/TrackerDevelopmentPlanning https://wiki.python.org/moin/TrackerDevelopmentPlanning to track everything that needs to be done on the Roundup side. If you prefer I can later move this to the new PEP, but for now I'm using it to keep track of all the things that come up in the various threads.
Nope, the wiki is fine for that sort of thing.
-Brett _______________________________________________ core-workflow mailing list core-workflow@python.org mailto:core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct https://www.python.org/psf/codeofconduct
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Tue, 5 Jan 2016 at 10:11 Donald Stufft
On Jan 5, 2016, at 1:03 PM, Brett Cannon
wrote: On Mon, 4 Jan 2016 at 21:54 Ezio Melotti
wrote: On Tue, Jan 5, 2016 at 2:42 AM, Brett Cannon
wrote: So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython.
On top of this, there is also the test repo (https://hg.python.org/test) and all the tracker repos (https://hg.python.org/tracker/). I think it would be useful to port the former since it will provide a place for devs to try things out and experiment (a new test repo could also be created though). It would be nice to port the tracker repos too and be consistent with the others, but it's not a priority. When we switched to HG they kept being on SVN until I ported them, so I guess the same thing can be done (unless R. David or Martin prefer to stick to HG).
I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
I think the current list of core-devs should be converted to a group and given access to the same repos they have access to now (i.e. cpython/devguide/peps and possibly others). Then additional repo-specific groups can be created in case we want to let specific contributors work on peps or the devguide.
This seems to be the general consensus, so we will create a python-dev team under the python org and add the core devs there.
Just to expand on this more, GitHub has recently updated their permission model more to also allow “outside contributors” to be added to a single repository without adding them to a group (Not sure if this would be useful for us or not).
It will be on the ancillary repos like peps and devguide where we may want to be more liberal in letting people commit (e.g., maybe we want to add all PEP authors as outside contributors so that they can maintain their own PEP).
I imagine the python-dev team would have “Write Access” (See https://help.github.com/articles/repository-permission-levels-for-an-organiz...) unless we think it needs admin access.
I think write access is perfect for the vast majority of people. The question becomes if we want to add anyone specifically with admin access to help manage repo stuff like web hooks, or if we just bug someone on the infrastructure team to set it for us. I do think that some of us -- probably those of us on hgaccounts -- should be team owners.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org?
This can be done. We can add a "GitHub" username field to Roundup users so that we can link the two.
OK, so it sounds like we will stick with our current CLA signing flow and write our own CLA bot that will query Roundup as to whether someone has signed the CLA or not and then throw up a banner signalling if someone has (not) signed and an appropriate link to the CLA. That will require some Roundup work and the creation of the bot.
We can set a commit status that will show red if the user hasn’t signed the CLA (just like if Travis tests failed or so). No need to use a banner or anything.
I should also mention, any bot creations we do should abstract out the code review tool so that when we change providers again in the future it will be more straight-forward to just update some select APIs rather than rewrite every bot we create.
Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
This also brings up another problem. Since the discussion about an issue happens on b.p.o and the PRs are submitted on GitHub, this means that: 1) users with only a GitHub account have to create a b.p.o account if they want to comment on the issue (exclusing review comments); 2) users with only a b.p.o account have to create a GitHub account if they want to review a PR; 3) users with both can comment on b.p.o and review on GitHub, but they might need to login twice.
It would be better if users didn't need to create and use two separate accounts.
If we can add GitHub as a login/creation option for b.p.o accounts then that solves that. But I'm willing to bet a majority of people will already have a GitHub account and we have always required the b.p.o account so #1 is the going to be the common case.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
As I mentioned on python-committers, we already have https://hg.python.org/lookup/ . This is currently used to map SVN->HG (e.g. https://hg.python.org/lookup/r12345 ), and should be extended to handle cs ids too. The b.p.o linkifier can just convert all revision numbers and cs ids to a https://hg.python.org/lookup/ link and let the lookup page figure out where to redirect the user.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
I think we should make hg.python.org read-only but keep it around and in sync with the GitHub repo (either via cronjobs or hooks). This will allow people to contribute using HG in the same way that the current GitHub clone allows people to contribute using git. It will also avoid breaking all the tools that currently use hg.python.org (and buys us more time to port them if/when needed).
That's easy to say, but someone also has to maintain hg.python.org then and we are doing this move partially to try and cut down on the amount of custom infrastructure that we maintain. If people are that worried about others being so adverse to using GitHub that they won't even do an anonymous clone from their servers then we can get a Bitbucket or GitLab clone set up, but I would rather try and cut out our repo hosting services if possible (who knows, maybe we can even finally retire svn.python.org thanks to shallow clones or something).
I think a mirror is a different level of involvement than running the main repository. We won’t have to worry about backups, if the server goes down people can still contribute, etc. Most likely it will also be able to be resized to a smaller server as well (but maybe not). I don’t think any of the infra team would be upset at one less server to manage though, so I’m more than happy either way.
Then I leave it to the infrastructure team to discuss amongst themselves to tell me if they think keeping hg.python.org up as a mirror is no big deal or if they simply want to set up some mirror at another host like Bitbucket or GitLab. Since it's your burden in the end it should be your decision as to whether you want to shoulder it to give people a purely open source way to contribute to Python (I also leave it in your hands whether making it a git or hg mirror is easier). -Brett
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
I think we should keep 2/3 together. We could split the stdlib from the rest, but that's a separate issue.
This seems to be the general consensus, so we will plan to keep cpython as a single repo.
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Master first and cherry-picking for older branches sounds good to me, but I don't know if switching model will have any implications, especially while going through the history or using tools like bisect.
This seems to be the general consensus, so we will plan to cherry pick commits into older branches.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
I would like to see a complete list of steps from starting to work on an issue to having it in the repo, at least to understand the new workflow. This doesn't have to include all the specific commands, but at least the basic steps (e.g. after I made a patch to I commit it and send a pull request to the main repo, or do I push it to my GitHub clone and push a button to send the PR? Do I need to create a branch before I start working on an issue?
There will be a step-by-step guide in the devguide to answer all of this before we make any switch.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
We could keep updating for regular patches with no related PR and add a note about all the other GIT contributors (possibly with a git command that lists all authors). Later on we might decide to have a script that automatically adds all the GIT contributors automatically.
This seems to be the general consensus, so we will keep Misc/ACKS around and have a tool that updates it based on git PR commits at release-time.
Six, we will need to update our Buildbot fleet.
If we keep hg.p.o around and updated, we might not have to do this now (even though now is better than never).
This gets us to the bare minimum needed to function.
Parity with hg.python.org ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org that the PR exists. If people disagree then feel free to say so.
FWIW I started adding notes to https://wiki.python.org/moin/TrackerDevelopmentPlanning to track everything that needs to be done on the Roundup side. If you prefer I can later move this to the new PEP, but for now I'm using it to keep track of all the things that come up in the various threads.
Nope, the wiki is fine for that sort of thing.
-Brett
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
On Jan 05, 2016, at 06:03 PM, Brett Cannon wrote:
If people are that worried about others being so adverse to using GitHub that they won't even do an anonymous clone from their servers then we can get a Bitbucket or GitLab clone set up
Once the migration settles down, I do plan on hooking up this GitLab mirror, which was original created as a PEP 507 experiment: https://gitlab.com/python-devs/python Cheers, -Barry
Day 1 summary
============
Decisions made
-----------------------
- We will create a python-dev team in the python org and add that team to
all of the relevant repos that get migrated
- We will add a GitHub field to Roundup and then write/get a CLA bot that
will query Roundup to check if someone has (not) signed the CLA
- For ancillary repos, the merge-of-least-resistance workflow is fine, so
no need to worry about maintaining a linear history
- We are going to keep all of the cpython history in a single repo
- We will have PRs go against master and then we will cherry pick into
bugfix branches
- Misc/ACKS will be kept and we will write code to update it on occasion --
probably at release time -- based on git commit data
- Seems like our current commit ID -> URL service can be updated to handle
our transition
Open issues
-------------------
- What tools and commands will we use to convert the repos?
- How do we want to handle Misc/NEWS?
- What are the exact commands we are going to expect core devs to run
(which will be documented in the devguide)?
- Who to use for CI (although Donald is the only one to speak up and likes
Travis as do I, so this might be decided)?
- Who to use for code coverage (Donald has suggested codecov)?
- Do we want to add GitHub login support for b.p.o to ease in adding
people's GitHub username and to minimize the need for two accounts to
propose a patch that has a corresponding issue?
On Mon, 4 Jan 2016 at 16:42 Brett Cannon
So consider this the starting discussion of the PEP that will be the hg.python.org -> GitHub transition PEP that I will be in charge of. Once we have general agreement on the steps necessary I will start the actual PEP and check it in, but I figure there's no point in have a skeleton PEP if we can't agree on the skeleton. :) While I list steps influencing all the repos, I want to focus on the ones stopping any repo from moving over for now, expanding what we worry about to the cpython repo as we knock blockers down until we move everything over and start adding GitHub perks.
The way I see it, we have 4 repos to move: devinabox, benchmarks, peps, devguide, and cpython. I also think that's essentially the order we should migrate them over. Some things will need to be universally handled before we transition a single repo, while other things are only a blocker for some of the repos.
Universal blockers ============== There are four blockers that must be resolved before we even consider moving a repo over. They can be solved in parallel, but they all need to have a selected solution before we can move even the devinabox repo.
First, we need to decide how we are going to handle adding all the core devs to GitHub. Are we simply going to add all of them to the python organization, or do we want something like a specific python-dev gteamroup that gets added to all of the relevant repos? Basically I'm not sure how picky we want to be about the people who already have organization access on GitHub about them implicitly getting access to the cpython repo at the end of the day (and the same goes for any of the other repos in the python organization). For tracking names, I figure we will create a file in the devguide where people can check in their GitHub usernames and I can manually add people as people add themselves to the file.
Second, CLA enforcement. As of right now people go to https://www.python.org/psf/contrib/contrib-form/, fill in the form, and then Ewa gets an email where she manually flips a flag in Roundup. If we want to use a web hook to verify someone has signed a CLA then we need to decide where the ground truth for CLAs are. Do we want to keep using Roundup to manage CLA agreements and thus add a GitHub field in bugs.python.org for people's profile and a web hook or bot that will signal if someone has the flag flipped on bugs.python.org? Or is there some prepackaged service that we can use that will keep track of this which would cause us to not use Roundup (which might be easier, but depending on the service require everyone to re-sign)? There's also the issue of supporting people who want to submit code by uploading a patch to bugs.python.org but not use GitHub. Either way I don't want to have to ask everyone who submits a PR what their bugs.python.org username is and then go check that manually.
Third, how do we want to do the repo conversions? We need to choose the tool(s) and command(s) that we want to use. There was mention of wanting a mapping from hg commit ID to git commit ID. If we have that we could have a static bugs.python.org/commit/<ID> page that had the mapping embedded in some JavaScript and if <ID> matched then we just forward them to the corresponding GitHub commit page, otherwise just blindly forward to GitHub and assume the ID is git-only, giving us a stable URL for commit web views.
Fourth, for the ancillary repos of devinabox, peps, benchmarks, and devguide, do we care if we use the GitHub merge button for PRs or do we want to enforce a linear history with all repos? We just need to decide if care about linear histories and then we can move forward since any bot we create won't block us from using GitHub.
Those four things are enough to move devinabox over. It probably is enough for the benchmarks suite, but I have an email to speed@ asking if people want to use this opportunity to re-evaluate the benchmark suite and make any changes that will affect repo size (e.g., use pip to pull in the libraries and frameworks used by a benchmark rather than vendoring their code, making the repo much smaller).
Website-related stuff ================ This also almost gets us the peps repo, but we do need to figure out how to change the website to build from the git checkout rather than an hg one. Same goes for the devguide. It would be great if we can set up web hooks to immediately trigger rebuilds of those portions of the sites instead of having to wait until a cronjob triggers.
CPython requirements ================= There are six things to work out before we move over cpython. First, do we want to split out Python 2 branches into their own repo? There might be a clone size benefit which obviously is nice for people on slow Internet connections. It also clearly separates out Python 2 from 3 and lets those who prefer to focus on one compared to the other do that more easily. It does potentially make any single fix that spans 2 and 3 require a bit more work since it won't be an intra-clone change. We could also contemplate sub-repos for things like the Doc/ or Tools/ directories (although I doubt it's worth it).
Second, do we want all fixes to go into master and then cherry-pick into other branches, or do we want to follow our current practice of going into the active feature branch and then merge into master? I personally prefer the former and I think most everyone else does as well, but I thought it should be at least thought about.
Third, how to handle Misc/NEWS? We can add a NEWS field to bugs.python.org and then generate the NEWS file by what issues are tied to what version and when they were closed. The other approach is to create a file per NEWS entry in a version-specific directory (Larry created code for hg already for this to give people an idea: http://bugs.python.org/issue18967). Then when we cut a release we run a tool the slurps up all of the relevant files -- which includes files in the directory for the next feature release which represent fixes which were cherry picked -- and generates the NEWS file for the final release. The per-file approach is bot-friendly and also CLI-friendly, but potentially requires more tooling and I don't know if people feel news entries should be tied to the issue or in the repo (although that assumes people find tweaking Roundup easy :).
Fourth, we need to decide exactly what commands we expect core devs to run initially for committing code. Since we agreed to a linear history we need to document exactly what we expect people to do for a PR to get it into their git repo. This will go into the devguide -- probably will want to start a github branch at some point -- and will act as the commands the bot will want to work off of.
Fifth, what to do about Misc/ACKS? Since we are using PRs, even if we flatten them, I believe the PR creators will get credit in the commit as the author while the core dev committing will be flagged as the person doing the merge (someone correct me if I'm wrong because if I am this whole point is silly). With the commits containing credit directly, we can either automatically generate Misc/ACKS like the NEWS file or simply drop it for future contributors and just leave the file for past contributors since git will have kept track for us.
Six, we will need to update our Buildbot fleet.
This gets us to the bare minimum needed to function.
Parity with hg.python.org ---------------------------------- For parity, there are some Roundup integrations that will be necessary, like auto-generating links, posting commits to #python-dev on IRC, etc. I'm not sure if people want to block until that is all in place or not. I do think we should make sure there is some web hook that can take an issue # from the title of a PR and automatically posts to the corresponding issue on bugs.python.org that the PR exists. If people disagree then feel free to say so.
Adding perks ========== Now we get to some added stuff that we never had on our own infrastructure. :)
We should wire up CI for all PRs. I don't know if we want to go with Travis, Codeship, or what CI provider, but we should definitely hook it up and fully utilize the resource. This could even include running doctest over the docs, making sure the reST markup is accurate, etc.
Do we need to set up a web hook to trigger website rebuilds? We should at least have a mirror on Read the Docs that is triggered by web hook so that we have a backup of the documentation (if possible; not sure how custom our Sphinx setup is compared to what they require to work).
We should try to get test coverage wired up as well per CI. I don't know if coveralls.io or some other provider is best, but we should see what is available and find out if we can use them to either get basic coverage or thorough coverage (read https://hg.python.org/devinabox/file/tip/README#l124 to see what thorough coverage entails, but it does require a checkout of coverage.py).
We should build a bot. It must use a Monty Python reference to trigger (black-knight, none-shall-pass, man-from-scene-24, five-questions, what-is-your-quest, what-is-your-favourite-colour, etc.; obviously I'm leaning towards the Black Knight or Bridge of Death scenes from the Holy Grail for inspiration since they deal with blocking you from doing something). It should handle specifying the commit message, what branches to commit to/cherry pick into, and a NEWS entry (if necessary). I don't know if it needs to do anything else as a requirement. It should probably implement a commit queue like Zuul or Homu (and both of those can be considered as the basis of the bot). Also gating commits on passing a test run probably would also be good.
I'm sure we will want to use some labels and milestones to track what PRs are for what versions, if they are blocked on something, etc.
---
Long email! :) I think that is my current brain dumped in email form. As I said at the beginning, I think we should focus on what is blocking the easiest repos first and then just keep knocking down blockers as we try to move over more repos.
On Tue, Jan 5, 2016 at 12:36 PM, Guido van Rossum
Why would you want this?
In general, our changes to 2.7 are separate from 3.x lines. So the pull requests and reviews could be handled separately. We could encourage feature contributions only to cpython (3.x) repo and any maintainance improvements from the folks who have 2.7 in production could be handled in 2.7 repo separately. Also, I was mapping our hg branches to git repos. Currently we have 3.4->3.5->default (active) And 2.7(active) -- Senthil
On Tue, 5 Jan 2016 at 12:35 Senthil Kumaran
Hi Brett,
On Tue, Jan 5, 2016 at 10:13 AM, Brett Cannon
wrote: - We are going to keep all of the cpython history in a single repo
Are we not going to have 2.7 as a separate repo? I believe, this is possible even if we keep cpython with full history (and branches).
No, we are not going to break 2.7 into a separate repo; everyone else wants to keep the cpython repo as-is with all branches in a single repo.
On Tue, Jan 5, 2016 at 8:13 PM, Brett Cannon
Day 1 summary ============
Decisions made ----------------------- - We will create a python-dev team in the python org and add that team to all of the relevant repos that get migrated - We will add a GitHub field to Roundup and then write/get a CLA bot that will query Roundup to check if someone has (not) signed the CLA - For ancillary repos, the merge-of-least-resistance workflow is fine, so no need to worry about maintaining a linear history - We are going to keep all of the cpython history in a single repo - We will have PRs go against master and then we will cherry pick into bugfix branches - Misc/ACKS will be kept and we will write code to update it on occasion -- probably at release time -- based on git commit data - Seems like our current commit ID -> URL service can be updated to handle our transition
Open issues ------------------- - What tools and commands will we use to convert the repos? - How do we want to handle Misc/NEWS? - What are the exact commands we are going to expect core devs to run (which will be documented in the devguide)?
I use the following commands for python.org: $ git pr NNN "git pr" is an alias and an equivalent of $ git fetch upstream pull/NNN/head:pr/NNN $ git checkout pr/NNN For example, if I want to review https://github.com/python/pythondotorg/pull/859, I use "git pr 859". Run tests, tweak commit message, code style, and commit $ git commit -m "Update foo bar" Squash commits $ git rebase -i HEAD~2 And merge $ git checkout master $ git merge --ff-only pr/NNN (--ff-only can be added to .gitconfig) Backport to 3.5 (the branch name is "release" for python.org) $ git checkout 3.5 $ git cherry-pick `git rev-parse master` And push to upstream $ git push --all upstream We probably should mention the -u option (or create a new section for Git configuration and best practices) in the devguide.
- Who to use for CI (although Donald is the only one to speak up and likes Travis as do I, so this might be decided)?
+1 to use Travis as a pre-merge CI. I'd vote for keeping Travis configuration simple (e.g. use only Linux and GCC, run rstlint in Doc/) --Berker
On Tue, Jan 5, 2016 at 11:13 AM, Brett Cannon
Day 1 summary ============
Decisions made -----------------------
Open issues -------------------
And a couple things that we are punting on: * code review tool (if GH proves undesirable) * separate (sub)repos for docs/tutorials--they could have a less restricted workflow than the rest of the cpython repo, a la the devguide Both of these can wait until later, though they still deserve mention in the PEP. -eric
On Tue, 5 Jan 2016 at 14:19 Eric Snow
On Tue, Jan 5, 2016 at 11:13 AM, Brett Cannon
wrote: Day 1 summary ============
Decisions made -----------------------
Open issues -------------------
And a couple things that we are punting on:
* code review tool (if GH proves undesirable)
Well, that's implicit if we find GitHub doesn't work for us for code review. I don't think it requires explicitly calling it out.
* separate (sub)repos for docs/tutorials--they could have a less restricted workflow than the rest of the cpython repo, a la the devguide
Sure, it can be mentioned.
Both of these can wait until later, though they still deserve mention in the PEP.
You really don't like GitHub's review tool, huh? ;)
We can set a commit status that will show red if the user hasn’t signed the
CLA (just like if Travis tests failed or so). No need to use a banner or
anything.
This is a great idea. Almost any automated check we want to run against PRs
can be captured as a Travis/CI test that shows up on the PR with its own
status https://github.com/blog/1227-commit-status-api.
Nick
On Tue, Jan 5, 2016 at 5:40 PM Brett Cannon
On Tue, 5 Jan 2016 at 14:19 Eric Snow
wrote: On Tue, Jan 5, 2016 at 11:13 AM, Brett Cannon
wrote: Day 1 summary ============
Decisions made -----------------------
Open issues -------------------
And a couple things that we are punting on:
* code review tool (if GH proves undesirable)
Well, that's implicit if we find GitHub doesn't work for us for code review. I don't think it requires explicitly calling it out.
* separate (sub)repos for docs/tutorials--they could have a less restricted workflow than the rest of the cpython repo, a la the devguide
Sure, it can be mentioned.
Both of these can wait until later, though they still deserve mention in the PEP.
You really don't like GitHub's review tool, huh? ;) _______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
2016-01-05 15:13 GMT-03:00 Brett Cannon
Open issues ------------------- - What tools and commands will we use to convert the repos?
I will investigate this soon. I don't claim ESR-level experience in repository conversion, but I did migrate most of KDE from SVN to Git (which was one hell of a task) :) However, I will be on vacation and trying to stay away from my email inbox (except urgent KDE sysadmin matters) from Jan 9th to Jan ~15th. What's the general timeframe you're thinking for all this? I know Python is not a corporate project with "deadlines", but I bet nobody would like "CPython will be in GitHub by early 2017"... -- Nicolás
Hi, On 01/05/2016 07:13 PM, Brett Cannon wrote:
Day 1 summary ============
Decisions made -----------------------
- Seems like our current commit ID -> URL service can be updated to handle our transition
Open issues ------------------- - What tools and commands will we use to convert the repos? - How do we want to handle Misc/NEWS? - What are the exact commands we are going to expect core devs to run (which will be documented in the devguide)? - Who to use for CI (although Donald is the only one to speak up and likes Travis as do I, so this might be decided)? - Who to use for code coverage (Donald has suggested codecov)? - Do we want to add GitHub login support for b.p.o to ease in adding people's GitHub username and to minimize the need for two accounts to propose a patch that has a corresponding issue?
a small question: could it be possible to access (some of) the services in a REST-ish form through a wrapper on https://www.python.org/dev I mean something like: Python- community- Service External-Service Python-Website-Address-Wrapper ==================================================================== cpython GitHub https://www.python.org/dev/cpython issues ... https://www.python.org/dev/cpython-issues website GitHub https://www.python.org/dev/website buildbots ... https://www.python.org/dev/cpython-buildbots devinabox GitHub https://www.python.org/dev/cpython-devinabox coverage codecov https://www.python.org/dev/cpython-coverage docs GitHub https://www.python.org/dev/cpython-docs ... Couldn't be then some bots external-provider independent/decoupled? Thanks in advance! francis PS: Sorry if I got some external-service wrong
On Thu, 7 Jan 2016 at 13:06 francismb
Hi,
Day 1 summary ============
Decisions made -----------------------
- Seems like our current commit ID -> URL service can be updated to handle our transition
Open issues ------------------- - What tools and commands will we use to convert the repos? - How do we want to handle Misc/NEWS? - What are the exact commands we are going to expect core devs to run (which will be documented in the devguide)? - Who to use for CI (although Donald is the only one to speak up and
On 01/05/2016 07:13 PM, Brett Cannon wrote: likes
Travis as do I, so this might be decided)? - Who to use for code coverage (Donald has suggested codecov)? - Do we want to add GitHub login support for b.p.o to ease in adding people's GitHub username and to minimize the need for two accounts to propose a patch that has a corresponding issue?
a small question: could it be possible to access (some of) the services in a REST-ish form through a wrapper on https://www.python.org/dev
I mean something like:
Python- community- Service External-Service Python-Website-Address-Wrapper ==================================================================== cpython GitHub https://www.python.org/dev/cpython
We will probably have git.python.org handle this.
issues ... https://www.python.org/dev/cpython-issues
bugs.python.org already exists.
website GitHub https://www.python.org/dev/website buildbots ... https://www.python.org/dev/cpython-buildbots
buildbot.python.org already exists.
devinabox GitHub https://www.python.org/dev/cpython-devinabox
It's just a repo and not special enough to deserve a unique URL.
coverage codecov https://www.python.org/dev/cpython-coverage
This might be worth a URL if we get it set up.
docs GitHub https://www.python.org/dev/cpython-docs
That's docs.python.org.
...
Couldn't be then some bots external-provider independent/decoupled?
The plan is that anything we write for this will be sufficiently decoupled from GitHub that changing hosts will just require updating abstraction code to work with different REST APIs and not a major rewrite of the bot itself. -Brett
Thanks in advance!
francis
PS: Sorry if I got some external-service wrong
_______________________________________________ core-workflow mailing list core-workflow@python.org https://mail.python.org/mailman/listinfo/core-workflow This list is governed by the PSF Code of Conduct: https://www.python.org/psf/codeofconduct
participants (16)
-
Barry Warsaw
-
Berker Peksağ
-
Brett Cannon
-
Donald Stufft
-
Eric Snow
-
Ezio Melotti
-
francismb
-
Guido van Rossum
-
Laura Creighton
-
M.-A. Lemburg
-
Nicholas Chammas
-
Nick Coghlan
-
Nicolás Alvarez
-
R. David Murray
-
Senthil Kumaran
-
Zachary Ware