On 21 March 2015 at 17:29, Jørn Lomax wrote:

On 21.03.2015 00:23, Ezio Melotti wrote:

...

On Fri, Mar 20, 2015 at 7:12 PM, Jørn Lomax wrote:

...

Hi

I'm a potential GSoC participant, and I'm interested in the bug tracker improvement idea[1]. I had a brief look at what has already been discussed on the mailing list and liked what I read.

I don't know anything about the code-review tools mentioned (Kallithea and Phabricator), but I would be up to investigating how they could be integrated. But I really like the idea of integrating a REST API to roundup, but I'm not sure if that would be a python-core project or a distinct roundup project (requireing mentors from the Roundup devs)?

Adding a REST API to Roundup would be a separate project, and it's currently being discussed on the roundup-devel ML (http://sourceforge.net/p/roundup/mailman/roundup-devel/). No one volunteered to mentor the project yet, but maybe someone will step forward if they see interested students.

Best Regards, Ezio Melotti

...

But I'm very interested in doing a project with the python core team, so if there are any other interesting ideas I'm definalty up to it. I should perhaps ask around on the python-dev ML?

[1]https://wiki.python.org/moin/SummerOfCode/2015/python-core

So is there any of the ideas discussed for bug.python.org you think might be a viable GSoC project?

I don't think we've discussed it anywhere yet (unless I mentioned it to Ezio on IRC), but there are some issues around dependency display that could conceivably be handled downstream. The main one is showing which bugs a given bug is *blocking* - that is, those which depend on the bug you're currently looking at. Another nice-to-have from my perspective would be the ability to add a new comment without having to scroll back to the top of the discussion. Unfortunately, I suspect those would also qualify as being projects best tackled under the Roundup banner, and I don't think it would be fair to ask a GSoC student to do that without a mentor that was already part of the upstream Roundup community that could facilitate getting their patches to a point where they were ready to be merged. Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote:

On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote:

...

I don't think we've discussed it anywhere yet (unless I mentioned it to Ezio on IRC), but there are some issues around dependency display that could conceivably be handled downstream. The main one is showing which bugs a given bug is *blocking* - that is, those which depend on the bug you're currently looking at.

Another nice-to-have from my perspective would be the ability to add a new comment without having to scroll back to the top of the discussion.

Unfortunately, I suspect those would also qualify as being projects best tackled under the Roundup banner, and I don't think it would be

Nope. Both of those are instance-only modifications, not core Roundup. The 'dependencies' field doesn't even exist in the stock roundup instance template.

Comment box at the bottom has been previously suggested and was rejected in favor of hotkeys that let you scroll back and forth easily. I think that decision could be revisited, though.

FTR, you can press 'r' (for reply) to jump to the reply box at the top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to the last message. There are other shortcuts that you can see by pressing '?' or by clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that these shortcuts only work in the issues pages.

Both of these are "small" modifications, so you'd need a bunch of additional items to make it a GSoC project. And, you need a mentor...that really seems to be the sticking point right now. I myself will simply have no time this summer to do it, otherwise I would.

I've been talking with a few people and this seems to be the situation right now: * There is the bugs.python.org project that I suggested and that I'm willing to mentor, and some students already showed interest (so I would advise them to also have a backup project in case they don't get accepted for this); * since there are several things that can be fixed/improved and none of them seems to have an particularly high priority, I think they can be decided together with the students (different students might prefer to work on different ideas in different order, and that shouldn't matter as long as they do enough work). * There is the Roundup project about adding a REST API that has been proposed upstream, however no Roundup mentor has stepped forward; * since some of the Roundup devs said they would be available to help, one option would be to find a core Python mentor for this project (Nick?) and then the student can still interact with the Roundup guys (including me) and get help from other devs as well. Best Regards, Ezio Melotti

--David

On 22.03.2015 08:50, Nick Coghlan wrote:

On 22 March 2015 at 14:11, Ezio Melotti wrote:

...

On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote:

...

On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote:

...

I don't think we've discussed it anywhere yet (unless I mentioned it to Ezio on IRC), but there are some issues around dependency display that could conceivably be handled downstream. The main one is showing which bugs a given bug is *blocking* - that is, those which depend on the bug you're currently looking at.

Another nice-to-have from my perspective would be the ability to add a new comment without having to scroll back to the top of the discussion.

Unfortunately, I suspect those would also qualify as being projects best tackled under the Roundup banner, and I don't think it would be Nope. Both of those are instance-only modifications, not core Roundup. The 'dependencies' field doesn't even exist in the stock roundup instance template.

Comment box at the bottom has been previously suggested and was rejected in favor of hotkeys that let you scroll back and forth easily. I think that decision could be revisited, though.

FTR, you can press 'r' (for reply) to jump to the reply box at the top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to the last message. There are other shortcuts that you can see by pressing '?' or by clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that these shortcuts only work in the issues pages. I almost never use hotkeys beyond Ctrl-C/Ctrl-X/Ctrl-V - if something doesn't have a GUI element associated with it, it generally doesn't exist as far as I'm concerned :)

I did just remember another small item though - hooking up searching by Real Name in addition to Username when adding someone to the nosy list (not everyone uses the first.last naming convention for their username, especially if they have an already established username they're bringing over from other services)

...

* since some of the Roundup devs said they would be available to help, one option would be to find a core Python mentor for this project (Nick?) and then the student can still interact with the Roundup guys (including me) and get help from other devs as well. I'm already trying to figure out how to gracefully hand at least some of my current projects over to other people, so I won't be volunteering for anything new any time soon :)

Cheers, Nick.

So as a student, if I'm writing a proposal to the project I should just mark it as bug tracker improvements and fixes? It just seem a little hard to point to very specific goals to complete, and the project would gain more goals and task goes by. I don't have any problem with it, I'm a very play by ear person so that works for me, I just don't want the project proposal to look to thin because not all the ideas are fleshed out yet Regards, Jørn Lomax

On Sun, Mar 22, 2015 at 9:50 AM, Nick Coghlan wrote:

On 22 March 2015 at 14:11, Ezio Melotti wrote:

...

On Sat, Mar 21, 2015 at 9:33 PM, R. David Murray wrote:

...

On Sat, 21 Mar 2015 23:13:13 +1000, Nick Coghlan wrote:

...

I don't think we've discussed it anywhere yet (unless I mentioned it to Ezio on IRC), but there are some issues around dependency display that could conceivably be handled downstream. The main one is showing which bugs a given bug is *blocking* - that is, those which depend on the bug you're currently looking at.

Another nice-to-have from my perspective would be the ability to add a new comment without having to scroll back to the top of the discussion.

Unfortunately, I suspect those would also qualify as being projects best tackled under the Roundup banner, and I don't think it would be

Nope. Both of those are instance-only modifications, not core Roundup. The 'dependencies' field doesn't even exist in the stock roundup instance template.

Comment box at the bottom has been previously suggested and was rejected in favor of hotkeys that let you scroll back and forth easily. I think that decision could be revisited, though.

FTR, you can press 'r' (for reply) to jump to the reply box at the top, 'esc' to unfocus it, and 'l' (lowercase L, for last) to jump to the last message. There are other shortcuts that you can see by pressing '?' or by clicking 'Keyb. shortcuts (?)' at the end of the sidebar. Note that these shortcuts only work in the issues pages.

I almost never use hotkeys beyond Ctrl-C/Ctrl-X/Ctrl-V - if something doesn't have a GUI element associated with it, it generally doesn't exist as far as I'm concerned :)

I did just remember another small item though - hooking up searching by Real Name in addition to Username when adding someone to the nosy list (not everyone uses the first.last naming convention for their username, especially if they have an already established username they're bringing over from other services)

That should already work, but the autocomplete only includes people in the expert list and committers.

...

* since some of the Roundup devs said they would be available to help, one option would be to find a core Python mentor for this project (Nick?) and then the student can still interact with the Roundup guys (including me) and get help from other devs as well.

I'm already trying to figure out how to gracefully hand at least some of my current projects over to other people, so I won't be volunteering for anything new any time soon :)

If you figure that out let me know, or else I'll have to take the cloning route again :)

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Thu, Mar 26, 2015 at 9:10 AM, Nick Coghlan wrote:

On 22 March 2015 at 23:16, Ezio Melotti wrote:

...

On Sun, Mar 22, 2015 at 9:50 AM, Nick Coghlan wrote:

...

I did just remember another small item though - hooking up searching by Real Name in addition to Username when adding someone to the nosy list (not everyone uses the first.last naming convention for their username, especially if they have an already established username they're bringing over from other services)

That should already work, but the autocomplete only includes people in the expert list and committers.

Aye, that part is great, the specific part I'm referring to is the search dialogue that pops up when you click on the "(list)" link.

I haven't really used that since I added the autocomplete -- I even considered removing it before forgetting about its existence. Once the REST API for Roundup is in place, these things can easily be improved (and new things can also be added -- e.g. searching related issues as you type the title of a new one).

I also wondered whether there might be a few enhancements that could be gathered under a "Improved mentoring support" theme, like being able to check how many patches someone has uploaded vs how many of the related issues have been resolved, or helping mentors keep a record of who's patches they've accepted recently.

This will likely require two changes: 1) add more metadata in the db (possibly determined automatically by Roundup/HG/Kallithea/etc.); 2) retrieving and presenting them to the user (this will be easier once the REST API is there). FWIW there are already plans about analyzing patches and adding to the DB the list of affected files and possibly other fields to signal if the patch contains tests and docs.

At the moment we can track commits fairly well, but there isn't really much of a link back to the uploaded patches outside the NEWS file and commit message, which automated tools tend not to read. (We could also look at using "hg commit -u" more to credit patch authors in the metadata, and something https://bitbucket.org/ede/committer/src/default/hgcommitter.py to track the committer info. On the other hand, it may not be worth doing that given the expected future work on forge.python.org based workflows, regardless of whether that's Kallithea or Phabricator based)

This is actually something that I wanted to bring up on python-dev. Currently our workflow is mostly patch-based, but adding support for pull requests from BitBucket/GitHub is one of the things that we are considering adding during GSoC. In addition, the GSoC students will work on a separate clone of the tracker, likely hosted on BitBucket. While we could still use the patch-based approach for GSoC, this would be a good chance to experiment with a more modern approach and also test on the meta-tracker both the new workflow and the code that will enable it. If successful, the same approach can also be adopted for CPython. One of the main issues is, as you mentioned, how to track both the committer and the author. This is both a technical and a "philosophical" issue -- that's why I wanted to bring it up on python-dev. Another issue is establishing a policy regarding branches and rebases (rebasements?). These issues might eventually be solved by Kallithea/Phabricator, but I expect a transition period where different workflows will be used. I would like our repo to be still intact by the time we settle on the new workflow. IIUC hgcommitter adds extra metadata to the changeset, and if we go down this route we might also consider adding metadata for the issue number and tweak hgweb to display both. Best Regards, Ezio Melotti

...

...

...

* since some of the Roundup devs said they would be available to help, one option would be to find a core Python mentor for this project (Nick?) and then the student can still interact with the Roundup guys (including me) and get help from other devs as well.

I'm already trying to figure out how to gracefully hand at least some of my current projects over to other people, so I won't be volunteering for anything new any time soon :)

If you figure that out let me know, or else I'll have to take the cloning route again :)

Taking into account the many things I'd like to eventually see fixed upstream when figuring out what I want to do with my professional career helps, but I can certainly see the attraction of the cloning option :)

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Thu, Mar 26, 2015 at 12:37 PM, Nick Coghlan wrote:

On 26 March 2015 at 19:05, Ezio Melotti wrote:

...

This is actually something that I wanted to bring up on python-dev. Currently our workflow is mostly patch-based, but adding support for pull requests from BitBucket/GitHub is one of the things that we are considering adding during GSoC. In addition, the GSoC students will work on a separate clone of the tracker, likely hosted on BitBucket. While we could still use the patch-based approach for GSoC, this would be a good chance to experiment with a more modern approach and also test on the meta-tracker both the new workflow and the code that will enable it. If successful, the same approach can also be adopted for CPython.

I managed to forget about the idea of allowing roundup patch generation direct from GitHub/BitBucket, so in that case it definitely makes sense to pursue some of this for the current bugs.python.org workflow.

Note that there are 3 distinct but related features here: 1) given a link to a separate clone/branch (e.g. on BitBucket/GitHub), compute the differences and create a diff file (this is already implemented for hg thanks to MvL); 2) add a button to commit patches directly from the bug tracker (this might happen only once we switch to Kallithea/Phabricator); 3) automatically detect/update pull requests against the official BitBucket/GitHub CPython mirrors. There are also two major approaches: 1) diff-based (we get a diff and apply/commit it ourself, we are the "user" in the cs); 2) changeset-based (we get a changeset and merge it, the contributor is the "user" in the cs); (user is the name of the metadata field used by HG to store the committer in the changeset.) For the second approach, the changeset can be provided either as a patch created with hg export, or a URL of a clone. In both cases I can either import/pull on my local clone and push to the main repo, or Roundup/Kallithea/Phabricator could do the same (possibly directly on the main repo). Switching to the second approach changes the semantics of the user field, since it will start representing the contributor, not the core-dev/committer. This will affect hg blame/annotate, and will also make things such as finding out how many patches have been committed by a given core-dev more difficult. If we decide not to change the semantics, we will have to edit the user field (assuming that's possible) and save the contributor name somewhere else.

It especially makes sense as both forge.python.org proposal target the support repos first (devguide, peps, etc), so regardless of which of those moves forward, it's going to be a long time before it impacts the CPython workflows. By contrast, improvements to Roundup's integration with other services can start being helpful as soon as they get deployed.

Note that currently nothing is preventing us to switch to the second approach, and some patches have already been imported with the contributor name as "user".

...

One of the main issues is, as you mentioned, how to track both the committer and the author. This is both a technical and a "philosophical" issue -- that's why I wanted to bring it up on python-dev.

I think there's plenty of precedent from the Git/Gerrit world here, but agree there should be a discussion to check for any disagreement with us following that precedent.

With git both fields are available, but not on mercurial (unless we use the extension you linked). This has already been discussed a few times in the past, see e.g.: https://mail.python.org/pipermail/python-dev/2011-November/114540.html

...

Another issue is establishing a policy regarding branches and rebases (rebasements?).

In terms of actually *make* changes, I think we'd still want changes to effectively involve apply patches until we're able to adopt a more capable repo hosting service with integrated review management.

While this is certainly the easiest way, it could also be possible to pull/import some changesets, tweak a few things (add Misc/ACKS and Misc/NEWS) and make a new commit (and possibly collapse/rebase as well). This doesn't need any new tool.

...

These issues might eventually be solved by Kallithea/Phabricator, but I expect a transition period where different workflows will be used. I would like our repo to be still intact by the time we settle on the new workflow.

Agreed.

...

IIUC hgcommitter adds extra metadata to the changeset, and if we go down this route we might also consider adding metadata for the issue number and tweak hgweb to display both.

The other possible direction to go is the direction Gerrit and the Linux kernel go, which is to just have a footer on commit messages for relevant key:value data fields, rather than using VCS metadata. This has the benefit of being easy to port to new VCS's later, since "committer & commit message" is setting the "what metadata does the VCS track?" quite low.

So your commit message might look like:

Apples are no longer counted as oranges

Apples were being counted as oranges in some cases. They're now always counted as apples.

Issue: 12345 Contributed-by: Pat Chauthor

This is also an option, but it requires everyone to be consistent. Having these in the commit message will also make them less accessible (including for the tool we will use to convert the repo to the next quantum VCS). Best Regards, Ezio Melotti

Cheers, Nick.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia