[core-workflow] bugs.python.org-related GSoC project

[Ezio Melotti]

On Mon, Mar 16, 2015 at 1:52 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 14 March 2015 at 04:39, Carol Willing <willingc at willingconsulting.com> wrote:
>> * Push button patch generation from a GitHub repo
>>
>> This looks like a well bounded project for a student.
>

I agree, however I think it could be completed in maybe a couple of
weeks, so it is not enough to be the only goal of the project.
For comparison,
https://hg.python.org/tracker/python-dev/rev/59532d2c180b seems to be
the equivalent code for Mercurial, so it's just a matter to do
something similar for git.

> It occurs to me also that this could potentially be done in a hosting
> service independent way by using Marcin Kuzminski's vcs module to
> actually generate the patch directly from the remote repo:
> https://pypi.python.org/pypi/vcs
>

This is also an option -- it would mean that instead of writing a git
equivalent of what I linked above, it would just need to be refactored
using vcs.
(FWIW the doc for vcs is at http://pythonhosted.org/vcs/quickstart.html)

> Added bonus: that library is the basis of the Git & Mercurial support
> in Rhodecode and Kallithea as well, so a patch generation utility
> based on it would potentially be useful for those projects as well.
>
> To avoid have to enter the full URL every time, we could potentially
> have something where a user could specify a CPython clone URL in their
> user preferences, and then prepopulate the VCS URL for patch
> generation based on that and a branch name based on the issue number
> like "issue12345".
>

This can be done, but I would say it's lower priority.

>> * Some tool that will update a checkout (or somehow make sure a clone is
>> clean for patching), grab a patch from an issue, apply it, run the test
>> suite, and then ask if the committer wants to commit the patch and submit it
>> (assuming everything else worked out in favour of committing the patch);
>>
>> This looks like a good student project with valuable experience for the
>> user.
>
> Pierre-Yves David (one of the Mercurial devs) has been working on a
> Mercurial extension for that at
> https://bitbucket.org/ncoghlan/cpydev/src/default/cpyhg.py?at=default
>
> He's hoping to spend more time on it soon so folks will have something
> to try out at the PyCon sprints, so I wouldn't bet on this idea still
> being around as a candidate project by the time GSoC rolls around.
>

I'm not sure if Brett was suggesting to do this on the client side
(i.e. a tool used by committers on their machines) or something on the
b.p.o side since both have been considered and discussed in the past.

If it's aimed to committers/contributors (like the one Nick linked)
then we have to see if people wants something similar. Personally I
find importing a patch from the tracker (hg imp --no-c
url_of_the_patch), running the tests (./python -m test), and
committing it (hg ci -m "message") trivial, so I would have little use
for this tool.  I might find it more interesting if it allowed me to
post patches to bpo without having to save a .diff and upload it
manually, or if it had some kind of interaction with the other tools
that we will use.

If it is intended to be used on the server side, then it might be more
interesting.  The basic idea is that when a patch is submitted, the
tracker will try to apply it on the branches specified on the
"version" field, and then compile / run the tests / build the doc
(depending on the content of the patch) and report back.  IIUC this
would be a simplified version of what Kallithea will do, so eventually
it might end up being replace.

>> essentially script what the fancy workflows being proposed using
>> Phabricator/Kallithea do with the assumption the code was already reviewed
>> in the issue tracker and deemed worthy of being committed
>
> Yeah, our general inspiration for the idea is actually OpenStack's
> git/Gerrit review plugin:
> https://github.com/openstack-infra/git-review
>
>>> Understanding in which direction we want to go will allow me to put
>>> together a project that, once completed, will have long-term benefits
>>> for our workflow.
>>> Perhaps I should post this to python-dev too and get feedback from a
>>> wider audience.
>>
>> If you want, but I would assume everyone who cares is here at least for an
>> initial discussion.
>
> I've found one neat trick for this kind of scenario is to devise
> standalone projects that are likely to be useful regardless of what
> happens in the broader context. REST-API-in-upstream-Roundup
> qualifies, and I think a vcs based utility library for easily
> generating patches from remote repos would likely also be useful,
> independently of its integration into our Roundup instance.
>

The problem with this is that creating a fully-featured general
purpose tool is much harder.  For example our Rietveld integration is
far from being mature enough to be included upstream, but it has been
serving us quite well for a few years now even though it's a
half-baked implementation.

Best Regards,
Ezio Melotti

>>> > Sure, we're likely to stop using Rietveld in favour of the winner of
>>> > the forge.python.org analysis at some point in the future, but that
>>> > point is likely to be quite some time away where CPython is concerned.
>>> >
>>>
>>> Having a student investigating how Kallithea and Phabricator will
>>> interact with Roundup and start developing a proof-of-concept
>>> integration and/or tools that we already know will be needed might
>>> also be an idea.
>>>
>>
>> Yes, especially if I can make a decision fast enough to know which one to
>> focus on.
>
> For the record, the reason it was fairly straightforward for me to
> wrangle some work time to spend on containerisation and related
> development workflow improvements for open source services like
> Kallithea was publicly announced last Friday:
> http://connect.redhat.com/zones/containers :)
>
> The relevant upstream bits are also there already
> (https://github.com/projectatomic/adb-atomic-developer-bundle) but
> there's still some work to do to on that front to integrate community
> Linux platforms in addition to RHEL.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia