Mailman 3 November 2015 - core-workflow

Software Factory
by Stefan Krah 31 Dec '15

31 Dec '15

[Brett] >> If GitHub turns evil, it will be most likely by selling this >> nicely packaged information to recruiters. It's too late to move >> away then, the statistics will have been collected already. > Do their terms of service even allow for that, or is this just speculation on your part? When exploring a worst case scenario, some level of speculation is probably inevitable. The guideline "if a service is free, you are the product" is pretty well established -- I cannot think of an example where it has actually been refuted. I do not know whether explicit permission to resell data that is technically public is required in the U.S. Note that while the data is "public", data mining GitHub by third parties seems to be forbidden. GitHub's TOS appear to be entirely silent on the subject, but they are already publishing aggregate data like https://github.com/torvalds , which by my (European) privacy point of view is already stretching and exceeding the limits of good taste. Stefan Krah

4 7

Other thoughts on the workflow
by Stefan Krah 01 Dec '15

01 Dec '15

Brett has asked for feedback until Dec 1st, so here are my quick random thoughts: 1) GitHub's UI is more polished than GitLab's. 2) GitHub pull requests are not universally liked (see famous Torvalds rant "... pull requests and the online commit editing, are just pure ..."). 3) Attracting contributors? This was also an argument for the svn -> hg change, but it did not happen IMO. On the contrary, patches like https://bugs.python.org/issue3526 seem to be more rare. 4) Personally, I probably would not have contributed to Python if a GitHub account had been required back in 2012. But the time may come when one is excluded entirely from any Open Source contributions without a GitHub account... 5) The "do-nothing" option hasn't been mentioned: The current setup is actually quite nice compared to almost 99% of other projects. I like it better than both GitHub/GitLab. Stefan Krah

6 10

Software Factory
by Stefan Krah 27 Nov '15

27 Nov '15

[Posting via gmane, followup seems to be broken.] [Nick] > From a purely pragmatic perspective though, "Import from GitHub" is > already one of the features offered by GitLab, and will no doubt be a > standard feature of any other open source repository management > systems that gains widespread popularity in the future. Donald also > noted a while back that this is a reasonable lesson to take away from > the Linux kernel Bitkeeper experiment - they used the proprietary > service for as long as the vendor was playing nice, and when the > relationship soured, they migrated away again. I think this is different from BitKeeper: GitHub has a dossier on every participating OSS developer, including contribution statistics, political correctness on the tracker, age estimate from pictures and so forth. Before GitHub, all this information *could* have been gathered on any individual, but at a far greater cost in time and effort. If GitHub turns evil, it will be most likely by selling this nicely packaged information to recruiters. It's too late to move away then, the statistics will have been collected already. Quite honestly, I'd feel like working for Andreessen & Horowitz for free if we move there. Stefan Krah

2 1

Software Factory
by Victor Stinner 25 Nov '15

25 Nov '15

Hi, As I wrote in my previous "I love OpenStack workflow" email, I really like the Gerrit tool. I have colleagues at Red Hat working on an integration of Gerrit, Zuul, Jenkins, Etherpad, etc. to easily deploy such setup (install but also update). The project is called "Software Factory" and it's a libre software: http://softwarefactory-project.io/docs/intro.html It would allow us (Python) to own our data (reviews), rather than relying on a private provider like Gitlab (I don't know if Gitlab can be used completly freely on our own server). I'm sure that my colleagues would love to help us to deploy a test setup for Python! I don't know if we need all Software Factory tools: Gerrit, Redmine, Zuul, Jenkins, Nodepool, Etherpad, Pastebin. If we keep Round Up, we will have to write code to integrate Round Up with Gerrit. IMHO it's worth it :-) By the way, I used Jenkins for different projects and different companies, and I prefer Jenkins over Buildbot. Its output (test result) is *much* better than Buildbot. It understands that some tests are unstable and identify them. I would also be a nice enhancement to modify our test suite to produce JUnit compatible output and get Jenkins reports. What do you think? Ok, I sent 3 emails. It's maybe time for me to read all the backlog (archives of the mailing list, PEP 507, etc.) :-D Victor

7 13

I love OpenStack workflow
by Victor Stinner 23 Nov '15

23 Nov '15

Hi, I'm working on the huge OpenStack project since 2 years. I really like its development workflow, even if it can be seen as complex at the beginning. The core of the workflow are reviews using Gerrit at review.openstack.org. In fact, I probably spend 80% of my time on review.openstack.org when I work on OpenStack. I probably spend the remaining 20% on IRC, the mailing and meetings to discuss. OpenStack uses Git and has a cool "git review" tool. It's very easy to send a review: commit something locally, type "git review", that's all! Your change will not be modified by others, you are reponsible to fix all issues, including all annoying coding style issues. But it's easy to modify a change. You can use local Git branches. But it's also very easy to retrieve a change into a new local branch: git review -d <review_identifier>. Since a change is a branch, retrieving a change does conflict. To rebase a change, use the common "git rebase master" command. Rebase is a common command in the OpenStack workflow. It's very easy to work on a patch serie: commit multiple changes locally, then type "git review". This common will create N reviews linked all together. For me, it's a real enhancement compared to the Python workflow. For example, it's common to start with a "cleanup/refactor" change and then a second change which does the "real" work (fix bug, add new feature, etc.). It's *much* easier to review a serie of small changes. On Gerrit, *anyone* can vote on a change. It's not used to ask for a bugfix. A vote means that the reviewer read the change and understand it. Votes: * +2: a core reviewer consider that the patch is good enough to be merged * +1: "looks good to me". A core reviewer may vote +1 to say "ok, but ..." with a comment * no vote: used to ask questions * -1: the patch must be updated to fix reported issues * -2: major design issue, you must rewrite your change differently, or the whole proposed change is rejected It's convenient to see a quick summary of votes. On Round Up, its requires to read individual comments, and almost nobody explicitly write "+1" or "-1" in comments on bugs.python.org. It's also possible to make a change depends on another. For example patch B depends on patch A. Even if the patch B is accepted, it will not be merged until the patch A is merged. Tests are run in pre-commit. There are 3 major kinds of tests: * coding style: basically, the flake8 command. Some projects use other tests (ex: pylint) * unit tests * functional tests It's very rare that coding style and unit tests produce "false positive". The functional tests are different. It's *common* to get random failures, it's almost "part of the process", even if it's very annoying. But it ensures that when a change is merged, it doesn't break anything. If it's no obvious: I would love to use the same workflow on CPython :-) Victor

1 0

My feedback on the Python workflow
by Victor Stinner 23 Nov '15

23 Nov '15

Hi, Brett contacted me to ask me to try the new workflow to develop Python. I'm sorry, I completly missed the whole discussiong. I would like to share my feedback on the Python workflow and the OpenStack worklow. I will start by listing random issues with the current Python workflow, even if you all probably know them (I hope). I'm contributing to CPython since more than 5 years. Since I started to work on OpenStack, I realized that how the CPython workflow is outdated. Attached .patch files to a bug tracker was cool 10 years ago, but today using merges is preferred. Issues with .patch files: * it's hard to "rebase" a patch: even if the patch contains the base revision, it's not convenient to rebase a patch (it's hard to rebasing a branch) * it's very hard to work on a patch serie, so nobody does that. As a consequence, patches can be very long and hard to review The integration between Round up (bug tracker) and Rietveld (review) is good, but incomplete. Adding comments to Rietveld doesn't copy them to the bug tracker. Unsubscribe from a bug doesn't unsubscribe from the the review, which can be very annoying for review staying a month or much longer (lot of spam). Because of the Misc/NEWS file, it's never possible to simply apply bindly a patch. It's common that I modify a patch after a review, to fix minor nits, like coding style. I don't want to annoy contributors with such minor details at the *end* of the review (when I missed them before). Tests are only run after the commit, it's common to break *all* buildbots because of a stupid mistake in a patch. The common issue is that modifying the module xxx requires to run test_xxx, but test_yyy fails and the review and the commiter forgot to run all tests. Another common issue is that a change breaks all Windows buildbots because of an obvious compilation isssue, but too few core reviewers test and work on Windows. I don't think that it would be possible to run the *full* test suite on pre-commit, but it would be a nice enhancement to start with a small subset of tests on buildbots known to be very stable (ex: only one Linux buildbot). And later slowly add more and more tests, and more and more buildbots. It's also annoying that Mercurial has no committer field, and so that the original author is not tracked in commit metadata. All core developers try to put the author in the commit message, but it's not counted in statistics. It would motivate to give more credits to contributors to "show" their name in commits. The commiter is also a very important information, it's the responsible of regressions, he/she has to break regressions and all side effects of the change. (Even if it's common and totally ok if someone else fixes these issues.) Victor

1 0