[Twisted-Python] Moving Twisted off Trac and SVN to somewhere nicer
Hi, As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN. A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the other hand, I do think that Trac is pretty universally loathed, and it would be a good idea to get away from it. There's a few existing hosting solutions: 1. Launchpad (+ Bazaar as the default vcs) 2. Bitbucket (+ Mercurial as the default vcs) 3. Github (+ Git as the default vcs) Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness). Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :) -- cheers lvh
On Fri, Jul 1, 2011 at 11:29 AM, Laurens Van Houtven <_@lvh.cc> wrote: ...
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I support Twisted moving to a DVCS and to something better than Trac. I personally would prefer Twisted to use Launchpad. Some points: * Launchpad is much faster now that it was six months ago * Its code review system works well with UQDS * Launchpad is open source & therefore patchable However, I won't argue too hard about it. jml
On Fri, Jul 1, 2011 at 12:42 PM, Jonathan Lange <jml@mumak.net> wrote:
I support Twisted moving to a DVCS and to something better than Trac. I personally would prefer Twisted to use Launchpad.
Some points: * Launchpad is much faster now that it was six months ago
Launchpad's definitely getting faster, but it's still not in the same ballpark. bzr, too, seems a lot slower on many operations (although I'm not sure how much of this is due to lp, and how much is due to git) despite having been sped up a lot over the years. Here's my incredibly unscientific (maybe these do significantly different amounts of work, I'm not sure) test for checking out Twisted using lp:twisted and github's powdahound/twisted: bzr: 63.29s user 2.80s system 1:43.32 total git: 6.93s user 3.28s system 0:45.75 total That data is skewed in bzr's favour because someone started downloading something huge halfway the git test. My point is that git's pretty fast, not even particularly so for clone (because the slowest part there is the network). Most of the work I've ever done on Twisted has been using bzr-svn, and it has been at times noticeably slow. (but not quite frustratingly so). Either way, I think this is probably the wrong discussion. Most of the time I hear "Launchpad's UX is bad for coders, and it's slow", not "Launchpad is slow and it's UX is bad". * Its code review system works well with UQDS
True, but I don't see how it works better than github's pull requests.
* Launchpad is open source & therefore patchable
Yes, that's a good point. I like that it's open source (although it has an unfortunate license). Is the patching of Launchpad by third parties followed by those patches landing in production a common occurrence? However, I won't argue too hard about it.
jml
cheers lvh
On Fri, Jul 1, 2011 at 2:08 PM, Laurens Van Houtven <_@lvh.cc> wrote:
On Fri, Jul 1, 2011 at 12:42 PM, Jonathan Lange <jml@mumak.net> wrote: ...
* Launchpad is open source & therefore patchable
Yes, that's a good point. I like that it's open source (although it has an unfortunate license). Is the patching of Launchpad by third parties followed by those patches landing in production a common occurrence?
Launchpad doesn't get a lot of external contributors, but their patches almost always land in production promptly. I suspect Launchpad is comparable to Twisted in this regard. <https://dev.launchpad.net/Contributions> miscategorizes some people, but gives some idea. jml
On Fri, Jul 1, 2011 at 3:17 PM, Jonathan Lange <jml@mumak.net> wrote:
On Fri, Jul 1, 2011 at 2:08 PM, Laurens Van Houtven <_@lvh.cc> wrote:
On Fri, Jul 1, 2011 at 12:42 PM, Jonathan Lange <jml@mumak.net> wrote: ...
* Launchpad is open source & therefore patchable
Yes, that's a good point. I like that it's open source (although it has an unfortunate license). Is the patching of Launchpad by third parties followed by those patches landing in production a common occurrence?
Launchpad doesn't get a lot of external contributors, but their patches almost always land in production promptly. I suspect Launchpad is comparable to Twisted in this regard.
Well, part of the hypothesis of the effects of moving to Github is that a) the clear separation between "core contributor" and "random contributor" because a bit more subtle, b) it becomes easier for external contributors to contribute. So yeah, I guess it is, but it'd be cool if it became a bit more open to contributions from the more general public :)
<https://dev.launchpad.net/Contributions> miscategorizes some people, but gives some idea.
Cool, thanks for the link. I had no idea there were that many.
jml
cheers lvh
On Fri, 2011-07-01 at 15:23 +0200, Laurens Van Houtven wrote:
Well, part of the hypothesis of the effects of moving to Github is that a) the clear separation between "core contributor" and "random contributor" because a bit more subtle, b) it becomes easier for external contributors to contribute. So yeah, I guess it is, but it'd be cool if it became a bit more open to contributions from the more general public :)
I'm not sure getting more patches should be our main goal, for now. (It's a good *long term* goal!). We have a large number of uncommitted third-party patches in tickets. We have a large number of half-finished developer branches (I'm working on a couple, since it's an easy way to get things done). These were not left uncommitted or unfinished because of tool problems, but because of other issues. Dealing with this seems to me to be higher priority than getting even more patches we won't get around to incorporating. If you want more contributions, improving the processes so abandonment is less likely to happen is the first step. I can certainly think of ways in which e.g. github might help with that, but this is not a *technical* problem, it's an organizational and social problem, and at the very least you should think about how to solve it before redoing all the technical infrastructure. For example, making sure all reviewable tickets get reviewed within 7 days, and all new tickets get an answer within 3 days. If a switch github is super-successful in getting us more patches, and then those patches sit in limbo forever, all we've done is alienate potential developers.
This, I believe, is the real problem -- tickets which were reviewed but never closed: http://twistedmatrix.com/trac/report/16 That is a very sad list.
Le 01/07/2011 15:44, Itamar Turner-Trauring a écrit :
On Fri, 2011-07-01 at 15:23 +0200, Laurens Van Houtven wrote:
Well, part of the hypothesis of the effects of moving to Github is that a) the clear separation between "core contributor" and "random contributor" because a bit more subtle, b) it becomes easier for external contributors to contribute. So yeah, I guess it is, but it'd be cool if it became a bit more open to contributions from the more general public :)
I'm not sure getting more patches should be our main goal, for now. (It's a good *long term* goal!).
At least personally, moving away from SVN and Trac is not to directly get more patches. It's mainly that I want as a contributor to use better tools. Also, I don't want us to maintain the infrastructure; for example, moving to a more recent Trac cost me personally a good amount of time; we also have that problem with spam.
We have a large number of uncommitted third-party patches in tickets. We have a large number of half-finished developer branches (I'm working on a couple, since it's an easy way to get things done). These were not left uncommitted or unfinished because of tool problems, but because of other issues. Dealing with this seems to me to be higher priority than getting even more patches we won't get around to incorporating.
If you want more contributions, improving the processes so abandonment is less likely to happen is the first step. I can certainly think of ways in which e.g. github might help with that, but this is not a *technical* problem, it's an organizational and social problem, and at the very least you should think about how to solve it before redoing all the technical infrastructure. For example, making sure all reviewable tickets get reviewed within 7 days, and all new tickets get an answer within 3 days.
If a switch github is super-successful in getting us more patches, and then those patches sit in limbo forever, all we've done is alienate potential developers.
Well, that logic is a bit flawed though: you're kind of saying that we shouldn't use a better tool because it may bring us more contributors than we can handle. At the end of the day, we would still use a better tool though. -- Thomas
Well, that logic is a bit flawed though: you're kind of saying that we shouldn't use a better tool because it may bring us more contributors than we can handle. At the end of the day, we would still use a better tool though.
No, I'm saying that given limited resources, addressing the giant piles of unused code we have (and figuring out how new code won't end up in the attic) seems like a higher priority. Using better tools is always a good idea, and if you'd like to argue that github means patches are less likely to be abandoned that's a good argument to make. I'm arguing for a broader look at what our development process problems are, and that perhaps efforts should be directed elsewhere.
Itamar makes an excellent point. That probably is one of the worst problems we have right now. and it definitely needs to be addressed. Figuring out how code could end up not-in-the-attic sounds like part of the transition plan to me. I do, in fact, think Github means patches are less likely to be abandoned. There's two reasons for that. First of all, you don't have to deal with a patch: you deal with a pull request. That means there's a branch with commits, and it's already in version control. Because that version control lives on Github, it's very easy to pull into your own checkout and work on it. This lowers the barrier to entry further, and completely gets rid of the distinction between committers and non-committers we currently have (where non-committers don't get to use version control, basically, unless they use bzr or something else, which again is a distinction between workflows for people with and without commit bits). The main problem I see in this transition is that Github tickets and pull requests are distinct elements in the issue tracker, whereas reviews go on the ticket in Trac/UQDS. cheers, lvh
In order to have at least some anecdotal evidence -- If you've submitted a patch to Twisted (or started a branch) and it never made it in, how did that happen? I imagine reasons might include a review request to write tests, redesign requests, getting distracted, "it works for me", design discussions that never got anywhere... What happened in your case? (I'd also like someone -- lvh? -- to go through report 16 and see if we can come up with a summary of how those tickets ended up where they are.)
twisted.positioning never got in because I never finish anyth
On 01/07/11 17:08, Itamar Turner-Trauring wrote:
In order to have at least some anecdotal evidence --
If you've submitted a patch to Twisted (or started a branch) and it never made it in, how did that happen? I imagine reasons might include a review request to write tests, redesign requests, getting distracted, "it works for me", design discussions that never got anywhere... What happened in your case?
I have submitted patches that went nowhere: http://twistedmatrix.com/trac/ticket/4602 http://twistedmatrix.com/trac/ticket/4603 http://twistedmatrix.com/trac/ticket/4610 http://twistedmatrix.com/trac/ticket/4666 Some of them seemed to be: "No, don't do it this way, do it that way". I'll be honest, that's just completely demoralising, when you're contributing new functionality. On that topic, I don't think the Twisted process is as accessible as some of you guys think it is I'm afraid. I found the discussion about the IPv6 tickets a bit disheartening :o( However, more constructively (less whiney!) some tickets languished in "make these tiny cleanups" and that's just incredibly painful in the current setup, with SVN and Trac mediating things. I've got absolutely no interest in pulling SVN head, writing a patch, submitting it as an attachment via Trac and *then* being told "ok, I've created this branch. Go off and learn how to do branches in a crappy old centralised VCS, and in a way compatible with UQDS, re-do your patch in a branch, then send another diff in as a file" Really? I mean, come on guys... If it were git/github, then I'd simply make the incremental changes in my local git branch, re-push to github and re-submit the pull request. I hate Launchpad, but I'm sure it's equivalent beats the crap out of SVN. Please, for the love of god, adopt a DVCS which lets contributors develop continuously against a local branch and push changes! Cheers, Phil
Hi all, On 01.07.2011 18:36, Phil Mayers wrote:
However, more constructively (less whiney!) some tickets languished in "make these tiny cleanups" and that's just incredibly painful in the current setup, with SVN and Trac mediating things.
I've got absolutely no interest in pulling SVN head, writing a patch, submitting it as an attachment via Trac and *then* being told "ok, I've created this branch. Go off and learn how to do branches in a crappy old centralised VCS, and in a way compatible with UQDS, re-do your patch in a branch, then send another diff in as a file"
I absolutely agree with Phil here. The twisted code and contribution standards are so high that patches from new contributors (like myself) are bound to be rejected/resubmitted at least once, maybe more. Don't get me wrong, I believe high standards are a good thing, but doing continuous development based on tools like svn and trac is really painful and it's really difficult to motivate yourself to work on a once rejected ticket. That being said, I believe that the move to a DVCS is a smart move for any project looking for continuous community contribution, because IMHO they simply allow for a more developer friendly process for all involved, thus making the whole review process a more friendly and less discouraging thing. Cheers, Chris
On Jul 1, 2011, at 1:08 PM, chris wrote:
doing continuous development based on tools like svn and trac is really painful and it's really difficult to motivate yourself to work on a once rejected ticket
Can you be more specific, please? What's painful? Procedurally, it's almost the same number of clicks (except for the unfortunate need to type the word 'review') to do this on Github or Launchpad. What part of the process is painful? If you're not a committer, we're not going to let you run code on our buildbots either way without a cursory review (that's just a recipe for automated attacks) so it's not like you get past that step for free, either. Plus, you can use the DVCS of your choice to actually author the patch.
On Jul 1, 2011, at 1:41 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Jul 1, 2011, at 1:08 PM, chris wrote:
doing continuous development based on tools like svn and trac is really painful and it's really difficult to motivate yourself to work on a once rejected ticket
Can you be more specific, please? What's painful?
I always found it especially irritating to come back to a patch later. If I don't already have a patched checkout of Twisted, I need to figure out what revision I was at before (or to actually be safe, make a HEAD checkout), then reapply my patch, hoping it is still valid. With a fork I can check it out any time, rebase to the current master (or branch I'm working on), having my changes reapplied for me. When I have made more changes I just push it up. No changes to tickets or switching keywords or watching Trac reject my patch file 10 times then clearing all my cookies or whatever. I've always admired Twisted's standards and process; I think they have made it possible for such a huge project to maintain working order for so long. The tools could use an upgrade, though.
Procedurally, it's almost the same number of clicks (except for the unfortunate need to type the word 'review') to do this on Github or Launchpad. What part of the process is painful? If you're not a committer, we're not going to let you run code on our buildbots either way without a cursory review (that's just a recipe for automated attacks) so it's not like you get past that step for free, either.
Plus, you can use the DVCS of your choice to actually author the patch.
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Jul 1, 2011, at 2:06 PM, Tom Davis wrote:
On Jul 1, 2011, at 1:41 PM, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
On Jul 1, 2011, at 1:08 PM, chris wrote:
doing continuous development based on tools like svn and trac is really painful and it's really difficult to motivate yourself to work on a once rejected ticket
Can you be more specific, please? What's painful?
I always found it especially irritating to come back to a patch later. If I don't already have a patched checkout of Twisted, I need to figure out what revision I was at before (or to actually be safe, make a HEAD checkout), then reapply my patch, hoping it is still valid.
With a fork I can check it out any time, rebase to the current master (or branch I'm working on), having my changes reapplied for me. When I have made more changes I just push it up. No changes to tickets or switching keywords or watching Trac reject my patch file 10 times then clearing all my cookies or whatever.
Wow, does that actually happen? The rejecting the patchfile, I mean. That's terrible. In any case, that should not be important. You _should_ be able to use the DVCS of your choice to work on Twisted.[1] As I said in the previous message:
Plus, you can use the DVCS of your choice to actually author the patch.
Lots of people do submit patches using Git; I reviewed one earlier this week. One thing I think this thread has inspired is a prompt move to make it clearer how to do this. It would be great if you could help out lvh in producing some good copy for the various workflow-documentation pages so that it's clear to people how to use their favorite VCS if they already have one; the SVN-based diff-and-ptach instructions that are already there are meant for users who may not be familiar with _any_ kind of version control. (Just to forestall any objections: this is a realistic audience, lots of Twisted contributors have been in secondary school when they started.)
I've always admired Twisted's standards and process; I think they have made it possible for such a huge project to maintain working order for so long. The tools could use an upgrade, though.
Thanks for that :). And my earlier post notwithstanding, I agree. In fact, the tools have improved significantly, but many of them are things that contributors don't see (build system upgrades, release process improvements) but core developers, and therefore eventually users, do benefit from. -glyph [1]: Your choice, of course, should be Bazaar, as all other choices are wrong. But we can work around your Git habit ;-).
Sent via iPhone Is your email Premiere? On Jul 1, 2011, at 11:41, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
Can you be more specific, please? What's painful?
Re-syncing whatever changes JP (just as an example of a reviewer) has made back into your local repo from SVN...which due to SVN's weakness on branch tracking makes merging fail frequently...then making your changes...generating a manual patch and finally uploading it to Trac. It would be far simpler to setup my DVCS to track JP's remote copy of my ticket's branch...then simply pull from that remote...make my changes and request he pull from me when he's ready to review. Automates the whole process quite a bit and reduces the round trip yak shaving. -J
It would be far simpler to setup my DVCS to track JP's remote copy of my ticket's branch...then simply pull from that remote...make my changes and request he pull from me when he's ready to review. Automates the whole process quite a bit and reduces the round trip yak shaving.
Any reason you can't do this with our git or bzr mirrors? http://twistedmatrix.com/trac/wiki/BazaarMirror http://twistedmatrix.com/trac/wiki/GitMirror Or for that matter, you can include e.g. an github URL in the ticket instead of attaching the patch.
On Fri, Jul 1, 2011 at 8:59 PM, Itamar Turner-Trauring <itamar@itamarst.org>wrote:
It would be far simpler to setup my DVCS to track JP's remote copy of my ticket's branch...then simply pull from that remote...make my changes and request he pull from me when he's ready to review. Automates the whole process quite a bit and reduces the round trip yak shaving.
Any reason you can't do this with our git or bzr mirrors?
http://twistedmatrix.com/trac/wiki/BazaarMirror http://twistedmatrix.com/trac/wiki/GitMirror
From a fresh checkout of the Git mirror:
lvh@hayek ~/tmp/Twisted ±master » git log -1 commit 936493c676c424f459bee93270f3a2870b68e8e5 Author: exarkun <exarkun@bbbe8e31-12d6-0310-92fd-ac37d47ddeeb> Date: Tue Mar 29 14:00:14 2011 +0000 ------------------snipsnipsnip------------------ Yeah. Mar 29? That's not quite right, is it?
Or for that matter, you can include e.g. an github URL in the ticket instead of attaching the patch.
Only if there's a decent Github mirror to fork from, otherwise you're asking people to do a multi-hour operation (I know, because I'm doing it right now) to get a decent git repo, followed by a large push to Github, and then maybe you can start doing some work. There is a seemingly good mirror maintained by powdahound, but it's not official and as such is not recorded anywhere. cheers lvh
On Fri, Jul 01, 2011 at 09:11:34PM +0200, Laurens Van Houtven wrote:
On Fri, Jul 1, 2011 at 8:59 PM, Itamar Turner-Trauring <itamar@itamarst.org>wrote:
Or for that matter, you can include e.g. an github URL in the ticket instead of attaching the patch.
Only if there's a decent Github mirror to fork from, otherwise you're asking people to do a multi-hour operation (I know, because I'm doing it right now) to get a decent git repo,
Last time I tried (perhaps a year ago), a git-svn clone of the Twisted SVN repo took the better part of a week. I seem to recall somebody preparing a tarball of a git-svn-clone'd repository to help people bootstrap, but my clone was already completed at that point so I didn't investigate further.
On Sat, Jul 2, 2011 at 02:23, Tim Allen <screwtape@froup.com> wrote:
On Fri, Jul 01, 2011 at 09:11:34PM +0200, Laurens Van Houtven wrote:
Only if there's a decent Github mirror to fork from, otherwise you're asking people to do a multi-hour operation (I know, because I'm doing it right now) to get a decent git repo,
Last time I tried (perhaps a year ago), a git-svn clone of the Twisted SVN repo took the better part of a week. I seem to recall somebody preparing a tarball of a git-svn-clone'd repository to help people bootstrap, but my clone was already completed at that point so I didn't investigate further.
I update the tarball a few times a year at http://ludios.net/mirror/ - see Twisted-checkout-README.txt for notes. If you do it yourself, keep in mind that git svn clone has to restart at r1 several times, for reasons I don't fully understand (due to partial SVN branches?). A few months back, a branch created by bzr with an svn:mergeinfo property caused it to restart at r1 again. This adds about 27 hours to the git svn clone time, unfortunately. Ivan
On Fri, Jul 1, 2011 at 12:59 PM, Itamar Turner-Trauring <itamar@itamarst.org
wrote:
It would be far simpler to setup my DVCS to track JP's remote copy of my ticket's branch...then simply pull from that remote...make my changes and request he pull from me when he's ready to review. Automates the whole process quite a bit and reduces the round trip yak shaving.
Any reason you can't do this with our git or bzr mirrors?
Because they don't always seem to track the ticket branch folders in a timely manner. Especially, when JP (he seems to usually be my reviewer :) ) pushes a modification of my patch to the ticket branch. It's at this point trying to merge in from SVN is usually a nightmare. My Git copy being tied to an older SVN rev that my patch is based on. SVN just seems to lose it's brains when my patch isn't in the SVN commit history, because SVN repo doesn't allow me to commit in. DVCS would allow me to branch, commit to my repo, and then let JP pull from my repo into his to review and push up to the Twisted repo when he's happy with it...and all of the commit history is sane from the original, to my patch to his changes, so when I go to pull back down from the Twisted repo everything merges sanely.
Or for that matter, you can include e.g. an github URL in the ticket instead of attaching the patch.
Honestly, the "process" documents are pretty clear you're supposed to attach a patch so that's what I do...also even that wouldn't solve the issue described above. -J
On Fri, Jul 1, 2011 at 6:01 PM, Jason J. W. Williams < jasonjwwilliams@gmail.com> wrote:
Because they don't always seem to track the ticket branch folders in a timely manner. Especially, when JP (he seems to usually be my reviewer :) ) pushes a modification of my patch to the ticket branch. It's at this point trying to merge in from SVN is usually a nightmare.
Why it is a nightmare? Just do svn checkout of "the ticket branch" and continue your work and submit additional patches against it if needed. The only problem here that I could see is if you have made some changes in addition to your patch. But in this case kdiff3 makes it a snap to merge you changes to the ticket branch checkout.
My Git copy being tied to an older SVN rev that my patch is based on. SVN just seems to lose it's brains when my patch isn't in the SVN commit history, because SVN repo doesn't allow me to commit in.
I can't decipher this, could you elaborate?
DVCS would allow me to branch, commit to my repo, and then let JP pull from my repo into his to review and push up to the Twisted repo when he's happy with it...and all of the commit history is sane from the original, to my patch to his changes, so when I go to pull back down from the Twisted repo everything merges sanely.
IMHO the common practice is to accept patches for review and potential inclusion and pull only from a trusted "lieutenants" (like in Linux kernel case) and creating patches is not very different in svn, git etc. Regards, Mikhail Terekhov
On Fri, Jul 1, 2011 at 12:41 PM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
On Jul 1, 2011, at 1:08 PM, chris wrote:
doing continuous development based on tools like svn and trac is really painful and it's really difficult to motivate yourself to work on a once rejected ticket
Can you be more specific, please? What's painful?
Procedurally, it's almost the same number of clicks (except for the unfortunate need to type the word 'review') to do this on Github or Launchpad. What part of the process is painful? If you're not a committer, we're not going to let you run code on our buildbots either way without a cursory review (that's just a recipe for automated attacks) so it's not like you get past that step for free, either.
For me the pain isn't Trac, it's SVN. The more I use DVCS's the more I hate it. Also, Combinator does not work on Windows, and hasn't for years. And before you say "submit patches", I did. They sat in the Divmod Trac instance for over a year, I requested reviews of the relevant tickets _daily_ for 3 months on Divmod's IRC channel, and they were never merged, or even reviewed, or even AFAIK put under version control. Now they're gone with the DIvmod site. If the tickets are ever recoverable, they were tickets #3001-#3004.
Plus, you can use the DVCS of your choice to actually author the patch.
In theory, yes, though it is not obvious how to do this in a way that is compatible with Twisted's workflow and Combinator. See http://twistedmatrix.com/trac/wiki/BazaarMirror for examples of some of the caveats of using DVCS on top of Subversion. Granted this works for just authoring a patch, which you then submit through Trac or whatever, but that doesn't really buy you a lot, IMO (though it does buy you something). Kevin Horn
On Fri, Jul 1, 2011 at 11:08 AM, Itamar Turner-Trauring <itamar@itamarst.org
wrote:
In order to have at least some anecdotal evidence --
If you've submitted a patch to Twisted (or started a branch) and it never made it in, how did that happen? I imagine reasons might include a review request to write tests, redesign requests, getting distracted, "it works for me", design discussions that never got anywhere... What happened in your case?
(I'd also like someone -- lvh? -- to go through report 16 and see if we can come up with a summary of how those tickets ended up where they are.)
I've had a few of these. I've also worked on others, reviewed a few, tried to resurrect some from abandonment, etc. Reasons include: - the finish line gets moved: A reviewer says "do this and then it'll be good to go into trunk". The contributor does "this" and then another reviewer says "oh just one more thing...". Rinse. Repeat. This is hugely demotivational. In many cases a better response from follow on reviewers would be to land the ticket, and then create another ticket for whatever other changes are needed. This is not always possible but I think it should be done where it is possible. - compatibility with some other unfinished ticket: The reviewer insists on compatibility with some other ticket, or waiting on some other ticket to land. This is not necessarily a bad thing, but if the other ticket takes 6 months to land...well, can we blame people for wandering off and/or forgetting about things? (to be fair, I think I've only seen this once, and I can't recall what it was or find the ticket at the moment). - I've seen a lot of tickets that "die" at the "this needs tests" phase. Requiring tests is a good thing, though. I think the only way to help this problem is for reviewers to provide more guidance as to how exactly to create those tests. Even figuring out where to put the tests is sometimes very difficult. Exarkun has been pretty good about this in the last year or so, IIRC. (It might be worth it to create a Trac keyword for this situation, maybe "needs-tests". This would make it easy to find those tickets, which might be a decent entry point for certain types of new contributors) - Some people just seem to wander off. - I can think of at least one ticket where the author stopped using Windows, and it was a Windows ticket. - That same Windows ticket is still open, though I tried to revive it. It basically tries to do 2 things. One of them is pretty nice (makes the StdIO protocol work on windows), but it's not even clear what the other one is trying to do. So the ticket is in limbo until someone has the time to either figure it out or split the ticket in two. - I recall one ticket that got stalled, because noone was entirely sure who exactly had written the patch. There was a third party who had contributed it to Twisted. This was easily resolved with a google search and an email. Of course the ticket is still open because it needs tests... - One ticket was stalled because it required a "merge forward". Which is not really obvious how to do, especially since the DIvmod site died. I'm sure there are dozens of other reasons, but these are cases I can recall off the top of my head. Kevin Horn
On 07/01/11 12:08, Itamar Turner-Trauring wrote:
In order to have at least some anecdotal evidence --
If you've submitted a patch to Twisted (or started a branch) and it never made it in, how did that happen? I imagine reasons might include a review request to write tests, redesign requests, getting distracted, "it works for me", design discussions that never got anywhere... What happened in your case?
I made it through the first several hurdles (working code, following coding standards, unit tests for everything) but then hit a legitimate reverse compatibility concern that kept my patches from landing. Someone eventually came up with a good solution, but the time gap meant that other things had changed and/or were about to change in Twisted, and gave more people a chance to bikeshed, which gave me less confidence that whatever I eventually finished would land. So I punted and waited for someone with more political clout to take over. Working with patches because you don't have svn commit rights is annoying, but this annoyance is a relatively minor fixed cost. The real issue, for controversial features, is achieving consensus, and then getting your feature in before consensus is lost. -- David Ripton dripton@ripton.net
On Jul 1, 2011, at 6:57 PM, David Ripton wrote:
Working with patches because you don't have svn commit rights is annoying, but this annoyance is a relatively minor fixed cost.
It's still important for us to reduce this cost; even if it's not the bottleneck, we have to optimize first where we can optimize :).
The real issue, for controversial features, is achieving consensus, and then getting your feature in before consensus is lost.
Yes, absolutely. And there's are some important guidelines for reviewers that can be inferred from this: Try to stick to coding-standard stuff as much as possible, especially if there's been a previous review. Don't bring up "I think it would be better if..." things, except to say "file an additional ticket". If there's a previous review, as much as possible, stick to the points brought up in the previous review. Make sure they're addressed, and try not to add a pile of conflicting stylistic suggestions. When you do a review, try to be as thorough as possible. Don't ever do a review that says "update @since markers" or "2 blank lines between methods" and nothing else; at least, you need to say "... and then it will be ready to merge". Remember that when you take it out of review, no other reviewer is going to look at it until the author fixes it and resubmits it, which may be quite a while. If you feel like adding some partial commentary to help the next full reviewer, just add a comment, don't remove the review keyword. Be explicit about what happens next, even if it's going to be redundant. Always say "... and it will need another review" or "... and then merge". Try not to voice a vague dissatisfaction with the architecture of something without an explicit suggestion about (A) what should be done, and (B) whether it needs to be done before the feature can be merged. For contributors, one suggestion: make implementation details private as much as possible, so that the reviewer will have to consider the aesthetics of the implementation details less. The smaller the public API of the contribution, the easier it is to avoid bikeshedding around method names and class placement :). None of this would have helped in particular on the IPv6 stuff, but given that that affected an extremely core API, and had a ton of fiddly little details, I'm not sure much could have helped on that one... I know I've broken these rules myself on occasion, and I'd like to encourage other reviewers to call me out on it when they notice it :). -glyph
On Sat, Jul 2, 2011 at 1:27 AM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
When you do a review, try to be as thorough as possible. Don't *ever* do a review that says "update @since markers" or "2 blank lines between methods" and nothing else
With Github's edit-this-file-on-the-web feature, it will effectively be easier to just fix it than it is to complain about how someone else needs to fix it. cheers lvh
On 1 Jul, 11:27 pm, glyph@twistedmatrix.com wrote:
On Jul 1, 2011, at 6:57 PM, David Ripton wrote:
Working with patches because you don't have svn commit rights is annoying, but this annoyance is a relatively minor fixed cost.
It's still important for us to reduce this cost; even if it's not the bottleneck, we have to optimize first where we can optimize :).
The real issue, for controversial features, is achieving consensus, and then getting your feature in before consensus is lost.
Yes, absolutely. And there's are some important guidelines for reviewers that can be inferred from this:
Try to stick to coding-standard stuff as much as possible, especially if there's been a previous review. Don't bring up "I think it would be better if..." things, except to say "file an additional ticket".
We've disagreed about this in the past, so I don't think you'll be surprised if I say that I don't think this is a good idea. :) If an earlier review misses *functional* issues with a change, then they need to be brought up. Scope creep should be avoided at *all* stages of the process, but an incomplete first review doesn't exempt a change from the development requirements (and I don't think you think it should, even though it sounds like you're saying it here :).
If there's a previous review, as much as possible, stick to the points brought up in the previous review. Make sure they're addressed, and try not to add a pile of conflicting stylistic suggestions.
Stylistic issues should all be known in advance (read the coding standard, etc) and brought up in the first review (because the first reviewer should know them too). Stylistic issues that aren't covered by the coding standard definitely shouldn't be sprung in a subsequent review (or the reviewer should address them himself or herself) - or even a first review, really. This is a separate case from pointing out *functional* issues that the first review missed.
When you do a review, try to be as thorough as possible. Don't ever do a review that says "update @since markers" or "2 blank lines between methods" and nothing else; at least, you need to say "... and then it will be ready to merge". Remember that when you take it out of review, no other reviewer is going to look at it until the author fixes it and resubmits it, which may be quite a while. If you feel like adding some partial commentary to help the next full reviewer, just add a comment, don't remove the review keyword.
This is very important, since it should reduce the instances in which a later review does have to introduce a new point. I don't think anyone benefits from forgoing resolving functional issues that are detected after the first review but before the change actually lands.
Be explicit about what happens next, even if it's going to be redundant. Always say "... and it will need another review" or "... and then merge". Try not to voice a vague dissatisfaction with the architecture of something without an explicit suggestion about (A) what should be done, and (B) whether it needs to be done before the feature can be merged.
For contributors, one suggestion: make implementation details private as much as possible, so that the reviewer will have to consider the aesthetics of the implementation details less. The smaller the public API of the contribution, the easier it is to avoid bikeshedding around method names and class placement :).
There are plenty of issues on the contribution-accepting side which I don't want to minimize, but I think there's another thing contributors can do to help the overall process. If a review results in more work than you're interested in doing, say so. Make it clear that you're no longer taking responsibility for the ticket. Then there's some chance that another contributor might take it the rest of the way (without waiting 5 years before deciding the original contributor has lost interest).
None of this would have helped in particular on the IPv6 stuff, but given that that affected an extremely core API, and had a ton of fiddly little details, I'm not sure much could have helped on that one...
Sooooooo fiddly aarrrrghhghhhhh.
I know I've broken these rules myself on occasion, and I'd like to encourage other reviewers to call me out on it when they notice it :).
This raises another point, which is that the mailing list isn't a terribly useful place for these points to end up. If anything is actually expected to change, then the need to update the review documentation (such as it is) and probably also get serious about meta- reviews. Jean-Paul
On Jul 1, 2011, at 9:06 PM, exarkun@twistedmatrix.com wrote:
On 1 Jul, 11:27 pm, glyph@twistedmatrix.com wrote:
On Jul 1, 2011, at 6:57 PM, David Ripton wrote:
Working with patches because you don't have svn commit rights is annoying, but this annoyance is a relatively minor fixed cost.
It's still important for us to reduce this cost; even if it's not the bottleneck, we have to optimize first where we can optimize :).
The real issue, for controversial features, is achieving consensus, and then getting your feature in before consensus is lost.
Yes, absolutely. And there's are some important guidelines for reviewers that can be inferred from this:
Try to stick to coding-standard stuff as much as possible, especially if there's been a previous review. Don't bring up "I think it would be better if..." things, except to say "file an additional ticket".
We've disagreed about this in the past, so I don't think you'll be surprised if I say that I don't think this is a good idea. :)
If an earlier review misses *functional* issues with a change, then they need to be brought up.
I don't think I disagree, actually. It's just a different emphasis. Issues certainly need to be brought up. I shouldn't have said otherwise. They just need to be clearly separated into blocking/not-blocking.
Scope creep should be avoided at *all* stages of the process, but an incomplete first review doesn't exempt a change from the development requirements (and I don't think you think it should, even though it sounds like you're saying it here :).
You're right, on a subsequent reading, that's what it sounds like I was saying. And I'm definitely not saying that; the review requirements are the same on every review.
If there's a previous review, as much as possible, stick to the points brought up in the previous review. Make sure they're addressed, and try not to add a pile of conflicting stylistic suggestions.
Stylistic issues should all be known in advance (read the coding standard, etc) and brought up in the first review (because the first reviewer should know them too). Stylistic issues that aren't covered by the coding standard definitely shouldn't be sprung in a subsequent review (or the reviewer should address them himself or herself) - or even a first review, really.
This is a separate case from pointing out *functional* issues that the first review missed.
The word "functional" is pretty broad, so let me be more specific. Any change which is part of the coding standard or the review policy needs to be mentioned. 100% test coverage is a requirement. Docstrings for public and private are a requirement. No matter how many reviewers have missed these things, they need to be brought up and no feature which is missing them may land. I think we are all pretty clear about that. Beyond that there's an infinite spectrum of diversity of possible review comments, and I can't speak to all of them. For example, maybe a change causes the performance characteristics of an existing, but un-benchmarked, change to regress. Maybe the API is designed in such a way that it will be difficult or impossible to scale. Perhaps something is treated as blocking where it should return a Deferred. I can't say for sure whether one of these issues would or wouldn't be a blocker without a lot of context about that specific change. But I think there's a general category of feedback that can be classified as "this is OK, but I can think of a way to do it better". In this case, a change might meet all the formal requirements, have a reasonable API that works, and not raise compatibility issues, but the reviewer has an epiphany while reading it and realizes it might be simpler/faster/more flexible to do something else. The reviewer should always feel free to communicate such an insight, of course, but I think that reviewers should all have a strong bias towards separating this out and making it optional - and making it clear that it's optional. The best should not be the enemy of the better; if it's an improvement, and it works today, we should usually integrate it. In many cases it's much better to defer the improvements until later and get the reasonable API in sooner. I think this is true even if the improved version will have a totally different API, because it's possible nobody will have a chance to do the improvements for years. Again, this is one that I know I'm guilty of, but I'd like to think that in recent reviews I've been really clear about the optionalness of my ideas :). The purpose of the review process, as I see it, is not to produce the best possible implementation of everything always. It's to produce an implementation that meets a certain quality standard and does not cause regressions (either in functionality or test coverage, although hopefully one day also in performance too). I think there are cases where we have ended up with no implementation rather than an adequate but not awesome implementation because there have been too many competing ideas for how to do it "even better" rather than how to make it meet the required standard. All that being said, I spent a long time looking at <http://twistedmatrix.com/trac/report/16>, and I can't find any tickets where comments like this are the only reason for blocking a ticket that's been through many round-trips; they all still have compliance issues. The larger problem is cursory early reviews where the patch is obviously bad, and later reviews that get more detailed once it starts to shape up. Apparently the more serious issue is that we just need to be more thorough earlier in the process. (Although another way to look at that data is that we weren't terribly thorough 3 years ago, since old tickets necessarily have old initial reviews.)
When you do a review, try to be as thorough as possible. Don't ever do a review that says "update @since markers" or "2 blank lines between methods" and nothing else; at least, you need to say "... and then it will be ready to merge". Remember that when you take it out of review, no other reviewer is going to look at it until the author fixes it and resubmits it, which may be quite a while. If you feel like adding some partial commentary to help the next full reviewer, just add a comment, don't remove the review keyword.
This is very important, since it should reduce the instances in which a later review does have to introduce a new point. I don't think anyone benefits from forgoing resolving functional issues that are detected after the first review but before the change actually lands.
If you're a new reviewer (or even if you're an old one who hasn't gotten much practice), it might be worthwhile to put a review comment on the ticket but ask a more experienced reviewer for a meta-review before getting rid of the 'review' keyword. A meta-review is usually a lot easier than a review, so it should be easier to get one by just showing up in #twisted and asking; hopefully we won't need an elaborate meta-review tracker. I hope this suggestion will make someone who has thus far been hesitant to do a review contemplate doing one with this set of training wheels.
Be explicit about what happens next, even if it's going to be redundant. Always say "... and it will need another review" or "... and then merge". Try not to voice a vague dissatisfaction with the architecture of something without an explicit suggestion about (A) what should be done, and (B) whether it needs to be done before the feature can be merged.
For contributors, one suggestion: make implementation details private as much as possible, so that the reviewer will have to consider the aesthetics of the implementation details less. The smaller the public API of the contribution, the easier it is to avoid bikeshedding around method names and class placement :).
There are plenty of issues on the contribution-accepting side which I don't want to minimize, but I think there's another thing contributors can do to help the overall process. If a review results in more work than you're interested in doing, say so. Make it clear that you're no longer taking responsibility for the ticket. Then there's some chance that another contributor might take it the rest of the way (without waiting 5 years before deciding the original contributor has lost interest).
+1
None of this would have helped in particular on the IPv6 stuff, but given that that affected an extremely core API, and had a ton of fiddly little details, I'm not sure much could have helped on that one...
Sooooooo fiddly aarrrrghhghhhhh.
I know I've broken these rules myself on occasion, and I'd like to encourage other reviewers to call me out on it when they notice it :).
This raises another point, which is that the mailing list isn't a terribly useful place for these points to end up. If anything is actually expected to change, then the need to update the review documentation (such as it is) and probably also get serious about meta- reviews.
I do hope someone will volunteer to update the wiki as a result, and if not, I hope I remember to. But the discussion has given me a couple of ideas I wouldn't have had otherwise; the meta-review comment above is a new(ish) idea. Of course, the most useful place for this to end up would be as a tweet! (Is everybody following @twistedmatrix yet?)
On 7/1/11 6:08 PM, Itamar Turner-Trauring wrote:
In order to have at least some anecdotal evidence -- I've had some patched rejected, probably on sound basis. But the experience always leave you with a feeling that you got stabbed.
Sometimes it _is_ be better to get some basic functionality in place, instead of arguing about how things should be done "the right way". From time to time you find a ticket that you want fixed, and start working on it, but end up never submitting it since you already know that even if it fixes the problem it will be shot down, since it doesn't do it this or that way. -- Johan Rydberg Product Designer Edgeware AB Mäster Samuelsgatan 56 SE-111 21 Stockholm, Sweden
On Tue, Jul 5, 2011 at 7:36 AM, Johan Rydberg <johan.rydberg@edgeware.tv>wrote:
On 7/1/11 6:08 PM, Itamar Turner-Trauring wrote:
In order to have at least some anecdotal evidence -- I've had some patched rejected, probably on sound basis. But the experience always leave you with a feeling that you got stabbed.
Yes, this is terrible. How can we fix it?
Sometimes it _is_ be better to get some basic functionality in place, instead of arguing about how things should be done "the right way".
From time to time you find a ticket that you want fixed, and start working on it, but end up never submitting it since you already know that even if it fixes the problem it will be shot down, since it doesn't do it this or that way.
Well, I think everyone would agree that a system where it's easier to pick up such a half-finished thing that just needs some love would be better, regardless of whether that unfinished work should go into twisted or not. Right? cheers lvh
On Jul 5, 2011, at 1:36 AM, Johan Rydberg wrote:
On 7/1/11 6:08 PM, Itamar Turner-Trauring wrote:
In order to have at least some anecdotal evidence -- I've had some patched rejected, probably on sound basis. But the experience always leave you with a feeling that you got stabbed.
We're always very grateful for contributions. I'm very sad to hear that you feel like you "got stabbed".
Sometimes it _is_ be better to get some basic functionality in place, instead of arguing about how things should be done "the right way".
Can you point to a specific ticket where you think this was the case? I have this same general feeling, but pretty much all of the reviews I found when I went looking for specific examples included at least some significant coding-standard, documentation, and test coverage problems. If we can find more specific examples, perhaps we can prevent this from recurring. I do agree that we don't want to block every ticket on the absolute best possible implementation; but, allowing changes that don't have test and documentation coverage is a recipe for creating an unmaintainable mess. Trust me, having dealt with Twisted in both modes, it's harder to get a patch into a release if the requirement is "demonstrate to everyone that it doesn't break everything without tests and documentation". It's just that it moves the work from you, the contributor, to a special hell that the release manager inhabits for months of painful pre-release debugging, or users who notice that all their applications are broken and file lots of bugs.
From time to time you find a ticket that you want fixed, and start working on it, but end up never submitting it since you already know that even if it fixes the problem it will be shot down, since it doesn't do it this or that way.
In these cases it would be best to file the ticket and describe your potential solution before implementing it. You can even put the 'review' keyword on it to indicate that you want feedback from a contributor before proceeding. -glyph
On 7/5/11 10:36 PM, Glyph Lefkowitz wrote:
Can you point to a specific ticket where you think this was the case? I have this same general feeling, but pretty much all of the reviews I found when I went looking for specific examples included at least some significant coding-standard, documentation, and test coverage problems. If we can find more specific examples, perhaps we can prevent this from recurring.
I was mostly thinking about the persistent connection functionality for twisted.web.client.Agent. Maybe Twisted would benefit more from having that functionality in place, than having the super-perfect API between Agent and HTTP parser.
I do agree that we don't want to block every ticket on the absolute best possible implementation; but, allowing changes that don't have test and documentation coverage is a recipe for creating an unmaintainable mess. I agree.
-- Johan Rydberg Product Designer Edgeware AB Mäster Samuelsgatan 56 SE-111 21 Stockholm, Sweden
On Wed, 2011-07-06 at 13:10 +0200, Johan Rydberg wrote:
I was mostly thinking about the persistent connection functionality for twisted.web.client.Agent.
We definitely want this to get in, this was a large part of the motivation for Agent in the first place.
Maybe Twisted would benefit more from having that functionality in place, than having the super-perfect API between Agent and HTTP parser.
The goal is not so much a perfect API as something we won't have to deprecate soon after because we realize there are some requirements that can't be addressed in a backwards compatible way. In this case, the cookie, proxy and other agent wrappers that have been created mean we now have a better understanding of what the Agent API looks like from a higher level, which should help.
On Jul 6, 2011, at 8:04 AM, Itamar Turner-Trauring wrote:
On Wed, 2011-07-06 at 13:10 +0200, Johan Rydberg wrote:
I was mostly thinking about the persistent connection functionality for twisted.web.client.Agent.
We definitely want this to get in, this was a large part of the motivation for Agent in the first place.
Looking at http://twistedmatrix.com/trac/ticket/3420 now, I see reviews with lots of functional issues and spec-compliance/correctness issues raised. There are a few notes on the API as well, but without addressing the reviews there, the patch would just have incorrect behavior. The few notes that are purely API aesthetics are mostly "make this private by default", which should be a trivial search-and-replace to fix.
Maybe Twisted would benefit more from having that functionality in place, than having the super-perfect API between Agent and HTTP parser.
The goal is not so much a perfect API as something we won't have to deprecate soon after because we realize there are some requirements that can't be addressed in a backwards compatible way. In this case, the cookie, proxy and other agent wrappers that have been created mean we now have a better understanding of what the Agent API looks like from a higher level, which should help.
That does sound a little like "the perfect API" to me :). And, I'd be inclined to argue - if I could find even one example of a functionally-correct patch with sufficient docs and tests that had actually been held up because of API issues :). -glyph
On Wed, Jul 06, 2011 at 05:27:55PM -0400, Glyph Lefkowitz wrote:
Looking at http://twistedmatrix.com/trac/ticket/3420 now, I see reviews with lots of functional issues and spec-compliance/correctness issues raised. There are a few notes on the API as well, but without addressing the reviews there, the patch would just have incorrect behavior.
This is also my experience. When fixing issue 4378, I received mostly constructive feedback. There is much knowledge how to "do things" when working on the codebase, some of it implicit or spread over multiple documents. During review, this knowledge was conveyed in a tangible way, using a concrete piece of code. I have to admit that it was a little discouraging to receive a long list of change requests. But the reviewers did a very good job explaining why things should be changed. This helped me to adapt my patches to fulfill the standards for the codebase. And it was very satisfying to finally have the patch accepted :-). Best regards, -- Albert Brandl Weiermayer Solutions GmbH | Abteistraße 12, A-4813 Altmünster phone: +43 (0) 720 70 30 14 | fax: +43 (0) 7612 20 3 56 web: http://www.weiermayer.com
Le 01/07/2011 16:42, Itamar Turner-Trauring a écrit :
Well, that logic is a bit flawed though: you're kind of saying that we shouldn't use a better tool because it may bring us more contributors than we can handle. At the end of the day, we would still use a better tool though.
No, I'm saying that given limited resources, addressing the giant piles of unused code we have (and figuring out how new code won't end up in the attic) seems like a higher priority. Using better tools is always a good idea, and if you'd like to argue that github means patches are less likely to be abandoned that's a good argument to make.
I'm arguing for a broader look at what our development process problems are, and that perhaps efforts should be directed elsewhere.
While I sympathize with what you're saying, this sounds kind of orthogonal to me. Sure, we can do better at handling contributions. But it's an opensource project, everybody does whatever he wants mostly (in the frame that the project sets). Saying to Laurens that we have limited "resources" and that he should do something else doesn't mean that this second task will be done. But it surely means that first one won't. -- Thomas
On Fri, Jul 1, 2011 at 6:24 PM, Thomas Hervé <therve@free.fr> wrote:
Well, that logic is a bit flawed though: you're kind of saying that we shouldn't use a better tool because it may bring us more contributors than we can handle. At the end of the day, we would still use a better tool though.
No, I'm saying that given limited resources, addressing the giant piles of unused code we have (and figuring out how new code won't end up in the attic) seems like a higher priority. Using better tools is always a good idea, and if you'd like to argue that github means patches are less
Le 01/07/2011 16:42, Itamar Turner-Trauring a écrit : likely
to be abandoned that's a good argument to make.
I'm arguing for a broader look at what our development process problems are, and that perhaps efforts should be directed elsewhere.
While I sympathize with what you're saying, this sounds kind of orthogonal to me. Sure, we can do better at handling contributions. But it's an opensource project, everybody does whatever he wants mostly (in the frame that the project sets).
Saying to Laurens that we have limited "resources" and that he should do something else doesn't mean that this second task will be done. But it surely means that first one won't.
-- Thomas
I'd say that's a safe bet. -- cheers lvh
On Jul 1, 2011, at 7:44, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
I'm not sure getting more patches should be our main goal, for now. (It's a good *long term* goal!). We have a large number of uncommitted third-party patches in tickets.
2 of those are mine. They're almost to the point of being accepted, but we're at that phase where there's a lot of roundtrips due to the code reviewing involved, and syncing SVN in that scenario is such a pain in the rear I've bogged down and gotten side tracked by other projects that have an easier system for contribution workflow ( Github or Bitbucket). I think you'd see the "wither on the vine" rate decline if Twisted moved to a system that made the workflow of contribution smoother. I like tools that make the work, rather than the tool, the focus of the job at hand...and that's definitely not SVN, Trac or LP. I'd advocate Git if only because it has the most number of integrations (that work well) with other VCSs, so devs can use the VCS/DVCS client they like best. I'd definitely vote for Github as well. The integration between the code, tickets and code review is by far the best I've worked with. Also, let's keep in mind the social aspect for visibility. When there's an interesting project, if it's on Github I'll hit the 'watch' button. This means I keep up-to-date on changes and activity for projects I want to use but may never contribute to. Github has opened my eyes to a number of projects that otherwise would have fallen off my radar...and I don't think that's an effect to dismiss lightly. -J
On Fri, Jul 1, 2011 at 2:23 PM, Laurens Van Houtven <_@lvh.cc> wrote: ...
Well, part of the hypothesis of the effects of moving to Github is that a) the clear separation between "core contributor" and "random contributor" because a bit more subtle, b) it becomes easier for external contributors to contribute. So yeah, I guess it is, but it'd be cool if it became a bit more open to contributions from the more general public :)
I'm sure the Launchpad project would gladly accept your help with getting new contributors... unless your help was to advise them to switch to Github ;) jml
On Fri, Jul 1, 2011 at 12:29 PM, Laurens Van Houtven <_@lvh.cc> wrote:
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I'm not sure I understand what you mean by "aren't devs". Do you mean "aren't Twisted developers"? I don't see why someone who isn't a developer would particularly care what development tools Twisted uses.
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
My personal preference would definitely be Launchpad+bzr > Bitbucket+hg > Github+git. There are also a fair number of Twisted-related projects already on Launchpad. On the other hand, I'm not sure this is a constructive way to approach this issue. -- mithrandi, i Ainil en-Balandor, a faer Ambar
On Fri, Jul 1, 2011 at 12:48 PM, Tristan Seligmann <mithrandi@mithrandi.net>wrote:
I'm not sure I understand what you mean by "aren't devs". Do you mean "aren't Twisted developers"? I don't see why someone who isn't a developer would particularly care what development tools Twisted uses.
Yes, this is what I meant.
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
My personal preference would definitely be Launchpad+bzr > Bitbucket+hg > Github+git. There are also a fair number of Twisted-related projects already on Launchpad.
Is the reason you would prefer bitbucket over github related to bitbucket and github, or git and hg? cheers lvh
On Fri, Jul 1, 2011 at 12:57 PM, Laurens Van Houtven <_@lvh.cc> wrote:
Is the reason you would prefer bitbucket over github related to bitbucket and github, or git and hg?
The latter. Launchpad+bzr is at the top of my list because of Launchpad; Bitbucket+hg ranks over Github+git because of hg. (Or because of git, depending on how you look at it...) -- mithrandi, i Ainil en-Balandor, a faer Ambar
On Fri, Jul 01, 2011 at 12:48:37PM +0200, Tristan Seligmann wrote:
On Fri, Jul 1, 2011 at 12:29 PM, Laurens Van Houtven <_@lvh.cc> wrote:
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I'm not sure I understand what you mean by "aren't devs". Do you mean "aren't Twisted developers"? I don't see why someone who isn't a developer would particularly care what development tools Twisted uses.
Perhaps "potential Twisted developers who haven't decided to contribute yet"?
On 1 July 2011 14:01, Tim Allen <screwtape@froup.com> wrote:
On Fri, Jul 01, 2011 at 12:48:37PM +0200, Tristan Seligmann wrote:
On Fri, Jul 1, 2011 at 12:29 PM, Laurens Van Houtven <_@lvh.cc> wrote:
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I'm not sure I understand what you mean by "aren't devs". Do you mean "aren't Twisted developers"? I don't see why someone who isn't a developer would particularly care what development tools Twisted uses.
Perhaps "potential Twisted developers who haven't decided to contribute yet"?
I'd support github. I'm somewhat new to twisted community, no code contributions so far (using twisted in my own projects), but github makes 3rd party commits to a project much easier. Ilja
On Fri, Jul 01, 2011 at 12:29:01PM +0200, Laurens Van Houtven wrote:
There's a few existing hosting solutions:
1. Launchpad (+ Bazaar as the default vcs) 2. Bitbucket (+ Mercurial as the default vcs) 3. Github (+ Git as the default vcs)
As a very-occasional Twisted contributor (but a long-time fan!) I'll vote for option 3. I'm not particularly a fan of Github[1], but Git is the DVCS I know best, and I'd be more than happy for Twisted Labs to make it easier for me (and people like me) to contribute. On the other hand, using git would probably complicate the build/review process: since Github repositories are (as far as I know) owned by individuals, you might not be able to set up access for multiple people, and hence the current scheme of "push your changes to a branch on the central server, tell the buildbots to build it" might not be possible. Tim. [1] In fact, I'm vaguely uneasy about "distributed version control" being so centralised on a single piece of commercially-owned infrastructure. On the other hand, the only hosted alternative would be Gitorious, and it doesn't have a lot of the shiny features Github has, like bug-tracking.
On Fri, Jul 1, 2011 at 12:59 PM, Tim Allen <screwtape@froup.com> wrote:
On the other hand, using git would probably complicate the build/review process: since Github repositories are (as far as I know) owned by individuals, you might not be able to set up access for multiple people, and hence the current scheme of "push your changes to a branch on the central server, tell the buildbots to build it" might not be possible.
No, this is not a problem, Github has the concept of "organisations" which Glyph has already set up, so we can have this done properly :) Also Github has well supported API notification support for when new commits come in. Not so much for issues, but I'm working on a solution for that...
Tim.
cheers lvh
Hi, On Fri, Jul 1, 2011 at 11:29 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I vote for GitHub too. Git's a pain but powerful (no need for combinator), and GitHub has a pretty good API into everything - low level repo innards, to issues/tickets. Also, GitHub's webhook system can easily integrate with buildbot (we used to do it at TweetDeck). Code review is doable by pull requests too. You can even map twistedmatrix.com to a github hosted website, which itself would be a repo. Oh, and the wikis are git repos too. Given all these tools, I see mappings for all of Twisted's bits and pieces (unless I'm missing something). Cheers, Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: rlotun@twitter.com @rlotun
Hi all, On Fri, 2011-07-01 at 12:00 +0100, Reza Lotun wrote:
On Fri, Jul 1, 2011 at 11:29 AM, Laurens Van Houtven <_@lvh.cc> wrote: Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
I vote for GitHub too. Git's a pain but powerful (no need for combinator), and GitHub has a pretty good API into everything - low level repo innards, to issues/tickets. Also, GitHub's webhook system can easily integrate with buildbot (we used to do it at TweetDeck). Code review is doable by pull requests too. You can even map twistedmatrix.com to a github hosted website, which itself would be a repo. Oh, and the wikis are git repos too.
Given all these tools, I see mappings for all of Twisted's bits and pieces (unless I'm missing something).
I agree, possibly the biggest win with GitHub is the way it encourages fellow users to fork a project and contribute patches via pull requests. Popularity doesn't always equate to quality but in this case (amongst developers) I think it *is* indicative that they've got something right. Pull requests can be used to implement the Twisted review model as they form a good centralised place to review a set of changes. GitHub's issue tracker used to be pretty shoddy but has had a big upgrade recently and is almost good now. Git is powerful, and while it can also be confusing at times StackOverflow almost always has the answer ;-) Also, managing branches in git really is real pleasure compared to the mish-mash of merging branches with SVN and various external scripts. For example, to merge 'somebranch' into master (i.e. trunk) and push it to GitHub (the 'origin' remote), after committing to somebranch, it's just: git checkout master; git merge somebranch; git push origin master Addressing the concern of getting your data out of GitHub, since git is a DVCS, every repository is a complete copy of the entire revision history. Therefore, GitHub cannot lock you in. I suppose the issue tracker might be a different story, but it has an API. -- Best Regards, Luke Marsden CTO, Hybrid Logic Ltd. Mobile: +447791750420 www.hybrid-cluster.com - Cloud web hosting platform
If the patch applies cleanly, pull requests can even be merged without involving git directly at all, from the Github web UI. cheers lvh
Le 01/07/2011 12:29, Laurens Van Houtven a écrit :
Hi,
As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN.
A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the other hand, I do think that Trac is pretty universally loathed, and it would be a good idea to get away from it.
There's a few existing hosting solutions:
1. Launchpad (+ Bazaar as the default vcs) 2. Bitbucket (+ Mercurial as the default vcs) 3. Github (+ Git as the default vcs)
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
Hi Laurens, Thanks for pushing this. Personally, I would prefer Launchpad and Bzr, but we've been thinking about using it for the past 3 years, and nothing really happened. What I really care about is that we move away from Trac (for bugs at least) and SVN. As I told you in Florence, if you come with a nice migration plan *and* you're ready to take on the burden, that's awesome and you have all my support for moving to Github. But, if somebody else is ready to do the same thing for Launchpad, I'll support him first. It seems unlikely though. -- Thomas
On Fri, Jul 1, 2011 at 12:16 PM, Thomas Hervé <therve@free.fr> wrote: ...
Thanks for pushing this. Personally, I would prefer Launchpad and Bzr, but we've been thinking about using it for the past 3 years, and nothing really happened. What I really care about is that we move away from Trac (for bugs at least) and SVN.
FWIW, I started trying to migrate to Bazaar a while ago, while still keeping us on Trac. (Back then, Launchpad was ruled out on account of being proprietary). It would be much easier to move to Bazaar *and* Launchpad at the same time. I do not believe that it has seriously been attempted by anyone involved in Twisted. jml
On Fri, 2011-07-01 at 12:29 +0200, Laurens Van Houtven wrote:
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to. And even then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github? In my opinion the biggest barrier to new developers is not whether we use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact. -Itamar
On Fri, Jul 1, 2011 at 2:14 PM, Itamar Turner-Trauring <itamar@itamarst.org>wrote:
On Fri, 2011-07-01 at 12:29 +0200, Laurens Van Houtven wrote:
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to.
Github has a much nicer API for pulling out all of your data than, say, Trac does. I know this because I'm drafting out sketches for getting something to port Trac's wikis and tickets to the other's...
...then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github?
git implies github because nobody, not even me, thinks git is worth using when you don't have github super powers to back it up. The point here isn't switching to git, it's switching to github. They've worked very hard to make contribution really easy to do, and I'd say they've succeeded. trac's git-plugin is not something I would work on transitioning, because part of the point is to get rid of Trac. Redmine *is* a much better system than trac (both in terms of maintenance and UX), but has other issues. For example, last time I tried the Bzr plugin for it it was pretty unusable. (Only supports a single branch? *Really*?). Redmine+git might be on the table but a) nobody really likes git that much b) we wouldn't get something as sexy as pull requests/merge proposals like we would on LP or Github. AFAICT there are no code review tools for Redmine that are quite as good. There are two code review plugins for Redmine. They've tried both at Markus, and the conclusion was "use review board". That gives us more moving parts to maintain. A huge advantage of using Github, IMHO, is the pull requests. You don't need to ask permission for a commit bit before you can start pushing to branches. You just fork and do your thing. This makes for a very low threshold for contribution. With Redmine, you'd still have to send patches in, which I think is a far less pleasant story than pull requests. (This paragraph applies equally well to Launchpad's merge proposals.) In Pull Requests, you see a timeline with both comments and commits, so it's very easy to check if comments passed in review have been addressed or not. In my opinion the biggest barrier to new developers is not whether we
use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact.
The reason I'd prefer Github is only partially the DVCS (and, like you said, any DVCS would do), but to a much larger extent the community on Github.
-Itamar
--
cheers lvh
Hi, On Fri, Jul 1, 2011 at 1:14 PM, Itamar Turner-Trauring <itamar@itamarst.org>wrote:
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to. And even then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github?
True, GitHub is proprietary, but it's free for open source projects. There are many high-quality open-source projects hosted on it: * Erlang/OTP (https://github.com/erlang/otp) * Redis (https://github.com/antirez/redis) * Jquery (https://github.com/jquery) * RabbitMQ (https://github.com/rabbitmq) * Ruby on Rails (https://github.com/rails) * Node.js (https://github.com/joyent/node) * Tornado (https://github.com/facebook/tornado) to name a few. GitHub also has an extensive API to programmatically access/backup all information around your project (like tickets and wikis), and it has integrated code review which allows you to comment on individual lines of code (this could be better, but generally works well). I suppose the main reason to even suggest it, given its proprietary nature and use of git, is that's currently where a lot of developer activity is, and its growing. GitHub makes it easy to contribute patches to projects and keep track of progress. It also has a nice way to keep track of related repositories (via 'organizations'). And, well, it looks better than Launchpad and is bit more mature than Bitbucket. Given the past involvement in Launchpad and its use of Twisted though, I understand why that would be an obvious choice. I just want to make sure all sides of the argument are represented. In my opinion the biggest barrier to new developers is not whether we
use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact.
I think you're right in that code quality is the most important factor here. However, as a small point, a DVCS surely should be able give the project a better ability to maintain high quality code though - isn't Combinator just tool over subversion to make branching easier? At least with most DVCS's branching is cheap, so there wouldn't need to be yet another tool to setup to get the development process started for new developers. Reza -- Reza Lotun mobile: +44 (0)7521 310 763 email: rlotun@gmail.com work: rlotun@twitter.com @rlotun
Le 01/07/2011 14:14, Itamar Turner-Trauring a écrit :
On Fri, 2011-07-01 at 12:29 +0200, Laurens Van Houtven wrote:
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to. And even then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github?
I think there are 2 sides though here: we want to move away from Trac as well as from SVN. And, the main reason for me is that we don't have to maintain Trac. So redmine wouldn't solve the whole problem. As Laurens said as well, the main attraction for git is using github. I don't think switching to git alone makes a ton of sense. One thing that concerns me is that Trac supposedly supports bzr, and we tried to use bzr, but never made the move completely. Why do you think it will change? Is it just that we didn't take any decisions? Or that nobody stepped up?
In my opinion the biggest barrier to new developers is not whether we use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact.
Well, even not thinking about new developers, a DVCS will make the life of current ones easier. Even though we can start to use bzr, it's a bit clumsy IMHO. -- Thomas
On Fri, Jul 1, 2011 at 3:23 PM, Thomas Hervé <therve@free.fr> wrote:
Well, even not thinking about new developers, a DVCS will make the life of current ones easier. Even though we can start to use bzr, it's a bit clumsy IMHO.
Also, apparently there are ways of using bzr-svn that confuse the hell out of Combinator or some other tools, and it'll spew metadata everywhere, or something. I don't remember what the conditions were, but I do remember that it was the obvious way of doing it, and glyph yelling at me for it after I tried ;-) cheers lvh
+1 for github. The user experience of bitbucket and LP is secondary compared to github.
On Fri, Jul 1, 2011 at 2:23 PM, Thomas Hervé <therve@free.fr> wrote: ...
One thing that concerns me is that Trac supposedly supports bzr, and we tried to use bzr, but never made the move completely. Why do you think it will change? Is it just that we didn't take any decisions? Or that nobody stepped up?
For me, the biggest external blocker was not being sure what the requirements were for moving. The internal blockers of time & motivation probably dominated. jml
On Fri, Jul 1, 2011 at 8:14 AM, Itamar Turner-Trauring <itamar@itamarst.org>wrote:
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to. And even then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github?
In my opinion the biggest barrier to new developers is not whether we use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact.
+1
-- Mikhail Terekhov
What data do you care about? Wikis and code are in git repositories, and they have a very exhaustive API[0] for pretty much everything else. On Sat, Jul 2, 2011 at 12:53 AM, Mikhail Terekhov <termim@gmail.com> wrote:
On Fri, Jul 1, 2011 at 8:14 AM, Itamar Turner-Trauring <itamar@itamarst.org> wrote:
Unless I'm mistaken, Github is a proprietary system, which means I'm unhappy about hosting our project there. At the minimum I'd want a very good story about how we can get all our data out if we need to. And even then I'd probably be against it. What's more, we can switch to git/bzr/hg without switching to a hosted system (e.g. trac with GitPlugin, and redmine has builtin integration for all of those.). Why does git imply github?
In my opinion the biggest barrier to new developers is not whether we use git or subversion or what have you, but the high quality of code required (coding standard, tests, passing code review). A DVCS may well encourage more users, but I'm skeptical it will have a major impact.
+1
-- Mikhail Terekhov
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[0] http://developer.github.com/ -- Jasper
On Fri, Jul 1, 2011 at 5:29 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Hi,
First, I encourage everyone to take a look at (and update) http://twistedmatrix.com/trac/wiki/WorkflowRequirements It was created a while back and hasn't been updated in a long time. Second:
1. Launchpad (+ Bazaar as the default vcs)
I'm curious about something: Is it yet possible to revert a merge in bzr without doing annoying things? -- Christopher Armstrong http://radix.twistedmatrix.com/ http://planet-if.com/
On Fri, Jul 1, 2011 at 5:36 PM, Christopher Armstrong <radix@twistedmatrix.com> wrote:
On Fri, Jul 1, 2011 at 5:29 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Hi,
First, I encourage everyone to take a look at (and update) http://twistedmatrix.com/trac/wiki/WorkflowRequirements It was created a while back and hasn't been updated in a long time. Second:
Launchpad (+ Bazaar as the default vcs)
I'm curious about something: Is it yet possible to revert a merge in bzr without doing annoying things?
I'm sitting around with a bunch of Bazaar developers, and they say this:: On trunk: bzr merge -r $REVNO..$(REVNO - 1) bzr commit The person with the failing branch, when they wish to resume development: bzr merge trunk bzr revert . bzr commit (I'm assuming the case where someone has committed a change that breaks the build and there may have been more commits to trunk since then.) jml
On 01/07/11 11:29, Laurens Van Houtven wrote:
Hi,
As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN.
A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the
As per my other, longer email: for me, the current SVN/branch development model is hugely offputting for occasional contributions, and I suspect for people who want to submit 10-100 line changes. For the love of god, pick a DVCS so that contributors can develop in local branches, and pick a DVCS with a good central "pull request" and patch review system/engine/site/app/whatever. I like github and loathe Launchpad personally but really - pick anything. Just not SVN. And not Trac.
On Jul 1, 2011, at 6:29 AM, Laurens Van Houtven wrote:
As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN.
A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the other hand, I do think that Trac is pretty universally loathed, and it would be a good idea to get away from it.
There's a few existing hosting solutions: Launchpad (+ Bazaar as the default vcs) Bitbucket (+ Mercurial as the default vcs) Github (+ Git as the default vcs) Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
As Itamar has already suggested, I think that this is a solution in search of a problem. Aside from occasionally saving a contributor the trouble of typing 'review' in the keywords field every so often, what is this actually going to accomplish? We have too many pending patches already, with not enough sustained interest to either review them or fix them; what we need are more sustained contributors, more reviewers. Is this going to help us sustain interest? In my opinion, the main frustration with this process is simply responsiveness of the site. I think a better investment to address those issues would be in faster hardware to run Trac on (and possibly to communicate the acceptability of using a DVCS mirror). But, if you're going to contemplate a transition, here are a few prerequisites. Perhaps some of this should be added to the WorkflowRequirements page for posterity. If you want me to take this effort seriously, please finish twisted.positioning first and get it reviewed and landed on trunk. That's just a patch, after all, a much smaller effort than a attempting to migrate the project onto entirely new infrastructure, with implications for testing, version control, and issue tracking. :-) Less controversial, I think, would be to produce a viable official github *mirror*, a-la <https://github.com/django/django>, rather than a migration, since this is a necessary prerequisite and useful in its own right. <https://github.com/twisted> still has no public repositories, and I've been bugging people (including you specifically, lvh) to take this over for the past couple of weeks. If we migrate somewhere, I would like all the data to be mirrored and presented on twistedmatrix.com with a custom stylesheet. Ideally, I would like to avoid Twisted's users interacting with a different site for issue tracking. A heavily-used, custom-styled web presence was an important part of fundraising. No, I don't have any strong empirical data to present to justify that sentiment, but salesmanship is an art, and when I was talking to potential sponsors I felt that our site made us look like a "serious" project that sponsors were comfortable contributing to. The only person who I think has done enough interacting with sponsors to credibly disagree with this is exarkun - although he may well do just that. For me, tens of thousands of dollars of focused sponsored development is going to win over random patches from people on github for a loooong time coming. The fact that Django still maintains their own Trac instance is a point in favor of this, as well. Those guys know a thing or two about image management :-). Of course, anyone may easily refute this point at any time by simply raising about $30,000 for the TSF. Pages like <http://www.sitemeter.com/?a=stats&s=sm7twistedmatrix&r=19> give us interesting information about our developers, and have previously informed discussions about efforts put towards platform support. None of the sites listed above appear to offer independent web analytics, or the ability to embed custom HTML. This is not, strictly speaking, a requirement, but any potential advocate of migrating somewhere should be aware that this would be one of the things we'd lose with a hosted solution. You'll need to port <http://twistedmatrix.com/highscores/> to whatever system we switch to, and hopefully also preserve the ease of development and experimentation with ticket data. We will need a staging deployment so that the new system can be evaluated and any tools to migrate can be tested without actually switching to it. This would appear to disqualify github immediately, as we can't run our own instance for testing, and I don't want people thinking that we've switched and starting to file bugs into a botched public test migration. I echo Itamar's concerns about a proprietary, remote code host. Even in the case of an open source host, there are practical considerations. I have the utmost respect for the developers at Launchpad, but the UI for that site is, frankly, a horror. Trac's bad, but it is at least simple, and fairly static, and reasonable to override when it's doing something silly, if you care enough. Once we've got all of our data ported over to Launchpad, I don't have a lot of confidence that the workflow or the UI will become _more_ suitable to our purposes, and if it's changed to become worse, will we really have any recourse? Will we have to update our project documentation? Our front page is already bad enough, but I think it still presents more useful information to the average reader than <https://launchpad.net/twisted>'s "Series and milestones", "configuration progress", or "packages in distributions" sections. Of course, you'll also need to rewrite all of the relevant wiki pages; ContributingToTwistedLabs, TwistedDevelopment, ReviewProcess. (And it took like 5 years to get this right for our current setup.) Finally, any code-hosting service should also have a good option for migrating away. We currently control twistedmatrix.com and can easily change the links from our current tickets to a new URL scheme or domain. But, when we decide that Github or Launchpad is no longer workable for whatever reason, will we be able to do the same with the ticket links under their control? We inhabit a hell now, to be sure, but it is a hell of our own design which we may hypothetically escape at any moment. I don't want to trade that for something slightly better that we could never leave. What are the non-hosted contenders? If trac is really so bad, where's the better, competitive software we could switch to? (Don't say redmine, I already know about that trick.) Sorry that this post is so long, but I've tried to incorporate the majority of my objections into this one post so that the discussion doesn't drag on too much. I am hoping to abstain from much more participation unless something is really unclear. My single biggest concern is the potential of this topic to waste a huge amount of core developer time, so I'll try to do my part and not fan the flames further. Although, the other questions raised in this thread, about what parts of our workflow are problematic, are worth talking about separately.
Although, the other questions raised in this thread, about what parts of our workflow are problematic, are worth talking about separately.
Moving from a manual patch-based process to a DVCS where branching is cheap and the workflow can be moved to push/pull would be a huge step in the right direction. Twisted has a very polite club-like culture where some are on the inside, most aren't and it's clear where on that line anyone is. Submitting to the pain of the current submission tools almost seems viewed as a kind of worthwhile hazing to weed out the unworthy. A lot of the resistance to change on this issue over the last year has had a lot of that flavor to it. Itamar's logic on why Twisted shouldn't make contributing easier being a prime example. Twisted needs more friends not fewer. Given everyone here has day jobs in addition to contributing to Twisted, getting rid of the patch-hell that is the current review process would help everyone even if Trac is kept for issue tracking. -J
Twisted has a very polite club-like culture where some are on the inside, most aren't and it's clear where on that line anyone is. Submitting to the pain of the current submission tools almost seems viewed as a kind of worthwhile hazing to weed out the unworthy. A lot of the resistance to change on this issue over the last year has had a lot of that flavor to it. Itamar's logic on why Twisted shouldn't make contributing easier being a prime example.
I did not say contributing shouldn't be easier; I said getting more contributions is pointless (and annoying to contributors!) if we can't manage to get them incorporated into trunk. If DVCS will make code easier to incorporate, by encouraging contributors to iterate on patches, then we should certainly do so. Given the feedback you and others have given that seems at least worth trying. There are other additional things we should work on though, e.g. giving better feedback to new developers.
I did not say contributing shouldn't be easier; I said getting more contributions is pointless (and annoying to contributors!) if we can't manage to get them incorporated into trunk. If DVCS will make code easier to incorporate, by encouraging contributors to iterate on patches, then we should certainly do so. Given the feedback you and others have given that seems at least worth trying.
My apologies if I misunderstood what you were saying. If we can make it easier to iterate that would address 95% of my current frustrations with the process of contributing.
There are other additional things we should work on though, e.g. giving better feedback to new developers.
No argument. But the Twisted code standards are high and usually have a high number of iterations to get any fix in...if we can get out of the stone age of shuffling patch files around and get to some form of push/pull system that would reduce a lot of my heartburn about "do this...OK looks good...now do this too and integrate with ticket XYZ...great almost there...I think you should do this now too...". -J
On 1 July 2011 18:38, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
As Itamar has already suggested, I think that this is a solution in search of a problem. Aside from occasionally saving a contributor the trouble of typing 'review' in the keywords field every so often, what is this actually going to accomplish? We have too many pending patches already, with not enough sustained interest to either review them or fix them; what we need are more sustained contributors, more reviewers. Is this going to help us sustain interest?
I agree that there is a problem getting patches reviewed, I think git provides a much better tool to reduce the effort and pain for a reviewer in applying patches, and updating branches to HEAD. Github's patch review tool, and the ability to get a diff to a patch reviewed are big wins in my opinion. I think github means less effort for the reviewer because they can review commits, rather than a large diff. They can review changes following a review. They can have a tool to make comments next to the code and it is less effort to apply patches and update branches to HEAD. Small, documentation diffs, for instance can be reduced to a single click for a core developer to merge the change to trunk. Michael
On Jul 2, 2011, at 1:07 PM, Michael Thompson wrote:
On 1 July 2011 18:38, Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
I think github means less effort for the reviewer because they can review commits, rather than a large diff. They can review changes following a review.
I already review diffs this way, with a local bzr-svn branch using 'bzr merge; bzr qlog'. Better yet I just sometimes do this on a plane :).
They can have a tool to make comments next to the code and it is less effort to apply patches and update branches to HEAD.
This would be nice. Is it easy for the reviewee to look at all the comments in an ordered list?
On Sat, Jul 2, 2011 at 11:27 PM, Glyph Lefkowitz <glyph@twistedmatrix.com>wrote:
They can have a tool to make comments next to the code and it is less effort to apply patches and update branches to HEAD.
This would be nice. Is it easy for the reviewee to look at all the comments in an ordered list?
Depends what ordering you want. Right now it's ascending line numbers, per
file they're in. You can also comment on an entire pull request if you want to say things like "3 lines between classes". -- cheers lvh
Michael Thompson wrote: [...]
Small, documentation diffs, for instance can be reduced to a single click for a core developer to merge the change to trunk.
I doubt that single click writes a NEWS file, or tests that the diff as applied to current trunk builds cleanly on a buildslave. So this may be less useful than you'd expect. -Andrew.
On Fri, Jul 1, 2011 at 5:29 AM, Laurens Van Houtven <_@lvh.cc> wrote:
Hi,
As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN.
A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the other hand, I do think that Trac is pretty universally loathed, and it would be a good idea to get away from it.
There's a few existing hosting solutions:
1. Launchpad (+ Bazaar as the default vcs) 2. Bitbucket (+ Mercurial as the default vcs) 3. Github (+ Git as the default vcs)
Although I've hated git for a long while (and I still don't like it very much), I firmly believe Github is the right thing for Twisted. My incredibly unscientific poll amongst people who like Twisted but aren't devs is that they all love or at least like Github, and a surprising number has a distaste for Launchpad (unfamiliarity with Bazaar, perceived developer-unfriendly UI, slowness).
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
-- cheers lvh
A bunch of thoughts after reading a lot of this thread: * any move to any DVCS will need a well-defined workflow/branching model/whatever, and it needs to be _written down_ so that people can be referred to it * moving to a DVCS would make it easier to merge changes from contributors * moving to a DVCS would allow contributors to work on their contributions in version control...no more patch nonsense. * though at times, after dealing with SVN/Combinator, I have longed for the patch nonsense... * A move to _any_ DVCS would almost certainly be a win over SVN. I've had to go back to using SVN at work for my day-to-day, and it is _painful_. * I've never had any huge problesm with Trac, though I can see that offloading admin responsibilities is a good thing. * Does Github's ticketing system have the kind of integration with Git that Twisted has built for Trac+SVN? Bitbucket has some similar stuff, so I would be surprised if Github didn't, but I just don't know. * I've never liked Launchpad. I find it confusing and hard to navigate. * Git is annoying. It's a pretty horrible piece of software in my opinion, and made some bad design decisions. * Git essentially makes Windows a second-class environment (as did Combinator). We already have trouble recruiting Windows people, and IMO one of Twisted's strengths is that you can usually run things on Linux/MacOSX/Windows with very few cahnges, if any. * Github almost makes up for Git's irritating-ness. * Github's extra features have never yet caused me to choose Git as a new project over Mercurial. * For me, Bitbucket has always been pretty comparable to Github. I know others will disagree, and point out all kinds of features that Github has over Bitbucket, but apparently I never use those features. Perhaps those features (whatever they are) are more important in projects of Twisted's size and/or history. * I have been told, but not actually experienced, that using the hg-git plugin for Mercurial makes using Git for your repository into "an implementation detail", and the user basicaly doesn't even really have to care that they aren't using Hg. If true, this would make me pretty happy...but I'm not entirely sure that it _is_ true. * it is unclear how Github would work with UQDS with regards to having a ticket for all new work. IMO this is even more important than code reviews. Would we just say: no pull requests will be accepted unless * How would running buildbot over a git/bzr/hg branch work? Would Github's hooks be able to drive this? Would we be able to tell buildbot to "go run tests on this branch"? * Assuming a change like this would take place, what would the plan be for transition? Would all the branches that are out there be migrate-able? What about all the Trac tickets, wiki, etc? * How does this affect the Sphinx transition plan? (I don't really know that it would affect it at all...). * Since Twisted's current web presence _is_ a Trac site, who's going to step up and build a new website? Github/Bitbucket/Launchpad is _NOT_ a replacement for an actual "marketing" or "home page" type site. Overall, this is probably a good plan, but it's going to bea harder and take more work than people think. Also, I agree with Itamar's comments that while switching to a DVCS might ameliorate Twisted's social/organizational issues somewhat, but is unlikely to wholly fix them. That stuff needs to be discussed and worked on as well. Kevin Horn
On Fri, Jul 1, 2011 at 8:00 PM, Kevin Horn <kevin.horn@gmail.com> wrote:
* Does Github's ticketing system have the kind of integration with Git that Twisted has built for Trac+SVN? Bitbucket has some similar stuff, so I would be surprised if Github didn't, but I just don't know.
Stuff like refs/closes interacts with the ticketing system yes. You can have as many hooks as you like for topfiles.
* Git is annoying. It's a pretty horrible piece of software in my opinion, and made some bad design decisions.
magit makes it a lot more pleasant if you use emacs, by the way.
* Git essentially makes Windows a second-class environment (as did Combinator). We already have trouble recruiting Windows people, and IMO one of Twisted's strengths is that you can usually run things on Linux/MacOSX/Windows with very few cahnges, if any.
I've been told this has *vastly* improved, but I don't know for sure since I'm not a Windows user.
* For me, Bitbucket has always been pretty comparable to Github. I know others will disagree, and point out all kinds of features that Github has over Bitbucket, but apparently I never use those features. Perhaps those features (whatever they are) are more important in projects of Twisted's size and/or history.
Yes, many features are comparable, the main feature differences are polish, but the killer difference is community size.
* I have been told, but not actually experienced, that using the hg-git plugin for Mercurial makes using Git for your repository into "an implementation detail", and the user basicaly doesn't even really have to care that they aren't using Hg. If true, this would make me pretty happy...but I'm not entirely sure that it _is_ true.
Although I've never tried this, I've been assured this is the case.
* it is unclear how Github would work with UQDS with regards to having a ticket for all new work. IMO this is even more important than code reviews. Would we just say: no pull requests will be accepted unless
Your question got cut off. I think you wanted to say "there's a ticket for that"?
* How would running buildbot over a git/bzr/hg branch work? Would Github's hooks be able to drive this? Would we be able to tell buildbot to "go run tests on this branch"?
Although I've never tried this, I've been assured this is the case.
* Assuming a change like this would take place, what would the plan be for transition? Would all the branches that are out there be migrate-able? What about all the Trac tickets, wiki, etc?
The first thing I thought right after I thought "we're going to need a trac to markdown converter" is "khorn is going to jump off a cliff next" ;-)
* How does this affect the Sphinx transition plan? (I don't really know that it would affect it at all...).
I don't see how it would.
* Since Twisted's current web presence _is_ a Trac site, who's going to step up and build a new website? Github/Bitbucket/Launchpad is _NOT_ a replacement for an actual "marketing" or "home page" type site.
Absolutely not. I am *not* advocating replacing this site, simply having
something similar that isn't trac and isn't involved in the site development. If anything, this separates the concerns of "home page style site" and development site even clearer, IMO.
Overall, this is probably a good plan, but it's going to bea harder and take more work than people think.
Also, I agree with Itamar's comments that while switching to a DVCS might ameliorate Twisted's social/organizational issues somewhat, but is unlikely
to wholly fix them. That stuff needs to be discussed and worked on as well.
There's no silver bullet. Not making new potential contributors bend over, and having all contributions trivially easy to put in version control, regardless of their contributor status, sounds like a good start.
Kevin Horn
cheers
lvh
On Fri, Jul 1, 2011 at 1:13 PM, Laurens Van Houtven <_@lvh.cc> wrote:
On Fri, Jul 1, 2011 at 8:00 PM, Kevin Horn <kevin.horn@gmail.com> wrote:
* Git is annoying. It's a pretty horrible piece of software in my opinion, and made some bad design decisions.
magit makes it a lot more pleasant if you use emacs, by the way.
I don't. Emacs is also annoying.
* Git essentially makes Windows a second-class environment (as did Combinator). We already have trouble recruiting Windows people, and IMO one of Twisted's strengths is that you can usually run things on Linux/MacOSX/Windows with very few cahnges, if any.
I've been told this has *vastly* improved, but I don't know for sure since I'm not a Windows user.
I've heard the same thing, but if what exists now is a _vast_ improvement, I shudder to think what it was like before.
* it is unclear how Github would work with UQDS with regards to having a ticket for all new work. IMO this is even more important than code reviews. Would we just say: no pull requests will be accepted unless
Your question got cut off. I think you wanted to say "there's a ticket for that"?
exactly.
* Assuming a change like this would take place, what would the plan be for transition? Would all the branches that are out there be migrate-able? What about all the Trac tickets, wiki, etc?
The first thing I thought right after I thought "we're going to need a trac to markdown converter" is "khorn is going to jump off a cliff next" ;-)
_almost_ funny :)
.
* Since Twisted's current web presence _is_ a Trac site, who's going to step up and build a new website? Github/Bitbucket/Launchpad is _NOT_ a replacement for an actual "marketing" or "home page" type site.
Absolutely not. I am *not* advocating replacing this site, simply having something similar that isn't trac and isn't involved in the site development. If anything, this separates the concerns of "home page style site" and development site even clearer, IMO.
Well, the site would _have_ to be replaced by something, if Trac were to go away. But I think I understand what you mean.
Kevin Horn
cheers lvh
Kevin Horn
On Fri, Jul 1, 2011 at 1:29 PM, Laurens Van Houtven <_@lvh.cc> wrote:
As some of you may already know (either through a backchannel or because you talked to me at Europython), there has been some talk about moving Twisted way from Trac+SVN to somewhere that isn't Trac+SVN.
There are always talks about moving here of there, because people are bored or it _seems_ to them that the other way of doing things is much better, because everybody else praises it. It is hard to resist this opinion. Some people even take it fundamentally - "I won't commit, because its not in Git". The worst that it doesn't mean these people will participate if the project is be in Git, but a small percentage will. Problem with Git/Bazaar/Mercurial and DVCS in general is much higher contribution barrier. So, if you want to switch, you need to know _exactly_ why, and more importantly - which features are you going to miss. See below:
A lot of the devs do like SVN. My guess is that that's mainly because they don't actually use SVN, they use Combinator, or something. On the other hand, I do think that Trac is pretty universally loathed, and it would be a good idea to get away from it.
I believe nobody uses Combinator, because it is dead http://divmod.org/trac/wiki/DivmodCombinator SVN has one major flaw - you can not stack patches on your system naturally when you don't have write access to repository. I believe that's the major complain behind "DVCS is better". Second problem - there in no _convenient_ way to share these patches to be reviewed and incorporated upstream. I do not think Trac is universally hated. It is the application one level above Python API as Twisted itself. Trac has its own architecture, different from standard OOP hierarchy. This architecture is not obvious and may be inconvenient to debug and extend. It may be that everybody is tired of Trac design, or Trac doesn't provide review and push/pull integration, but it has some other awesome features.
There's a few existing hosting solutions:
Launchpad (+ Bazaar as the default vcs) Bitbucket (+ Mercurial as the default vcs) Github (+ Git as the default vcs)
I am not sure if the following possible with these services: L G B [ ] [ ] [ ] Project timeline with changes to wiki, tickets, commits etc, [ ] [ ] [ ] Editable issue description (Google Code suxx at this) [ ] [ ] [ ] Commit history navigation from patch view (next/prev buttons) [ ] [ ] [ ] Colored blame history browser [ ] [ ] [ ] Hook scripts for bots and other stuff [ ] [ ] [ ] Full project data export
Unless someone is going to go all "NO GITHUB IS TERRIBLE AND YOU ARE A BAD PERSON FOR EVEN SUGGESTING IT" on me, maybe we can talk about planning the transition? :)
To know if Github is terrible or not, you need some data - examples, use cases. The first step in planning is to look at the current workflow and gather a list of ways current Trac+SVN is used and see where Github has advantages and where it suxx. Usually, people realize the latter when it's too late. As we are all mostly too busy, if you want people to participate in discussions, it will be better to outline the features you need from development workflow and separate discussion with some summary about them into separate thread. Right now I see that there is a thread about reviews and as a Rietveld user, I may have a lot to say about that. =) -- anatoly t.
On Tue, Jul 05, 2011 at 09:41:12AM +0300, anatoly techtonik wrote:
To know if Github is terrible or not, you need some data - examples, use cases. The first step in planning is to look at the current workflow and gather a list of ways current Trac+SVN is used and see where Github has advantages and where it suxx. Usually, people realize the latter when it's too late.
As has been mentioned in earlier in this thread: http://twistedmatrix.com/trac/wiki/WorkflowRequirements (which I have updated with some of the website requirements that Glyph mentioned in one of his posts).
Pretty much all of those can be supported with GitHub: they can POST to a generic website as a commit hook[0], along with a number of other integrated services[1]. The only thing that I can think of is that GitHub issues doesn't have hooks, so we'd have to poll if we wanted an IRC bot for GitHub issues. Thankfully, they have an API for issues[2] that should make it easier. re: Mercurial, I didn't like it when I used it. If someone can tell me how to do this[3] in hg, I'd be more inclined to play along. And that said, I think we'd get a much better reception and amount of contributors if we're on GitHub, if only due to the scale compared to LP/BB. I think we're all familiar with the "denied" story :). On Tue, Jul 5, 2011 at 2:49 AM, Tim Allen <screwtape@froup.com> wrote:
On Tue, Jul 05, 2011 at 09:41:12AM +0300, anatoly techtonik wrote:
To know if Github is terrible or not, you need some data - examples, use cases. The first step in planning is to look at the current workflow and gather a list of ways current Trac+SVN is used and see where Github has advantages and where it suxx. Usually, people realize the latter when it's too late.
As has been mentioned in earlier in this thread:
http://twistedmatrix.com/trac/wiki/WorkflowRequirements
(which I have updated with some of the website requirements that Glyph mentioned in one of his posts).
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[0] http://help.github.com/post-receive-hooks/ [1] https://github.com/github/github-services [2] http://developer.github.com/v3/issues/ [3] http://people.gnome.org/~federico/news-2008-08.html#git-rebase-interactive -- Jasper
On Tue, Jul 05, 2011 at 03:42:04AM -0400, Jasper St. Pierre wrote:
re: Mercurial, I didn't like it when I used it. If someone can tell me how to do this[3] in hg, I'd be more inclined to play along. And that
I do this sort of things using mercurial queues. I pile up patches in a queue and can subsequently navigate in the queue (hg qgoto fix_header1) and fold it with a later one (hg qfold fix_header2). While the queue is not yet committed I can change the commit log of a patch in a simple way. hg qnew -f fix1 -m "this fixed issue 1" hg qnew -I debian/control -m "fix control" hg qnew -f fix1.1 -m "forgot something in issue 1" hg qgoto fix1 hg qfold fix1.1 # This concatenate the 2 comments" hg qrefresh -e # fix your comment as you like it hg qpush hg qfinish -a # commit all queues currently applied sandro *:-) -- Sandro Dentella *:-) http://www.reteisi.org Soluzioni libere per le scuole http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy
On Tue, Jul 5, 2011 at 5:02 AM, Alessandro Dentella <sandro@e-den.it> wrote:
On Tue, Jul 05, 2011 at 03:42:04AM -0400, Jasper St. Pierre wrote:
re: Mercurial, I didn't like it when I used it. If someone can tell me how to do this[3] in hg, I'd be more inclined to play along. And that
I do this sort of things using mercurial queues. I pile up patches in a queue and can subsequently navigate in the queue (hg qgoto fix_header1) and fold it with a later one (hg qfold fix_header2).
Hm. So it's like quilt? Are patch queues real commits (changesets, revisions, whatever), so I can log and blame and grep them while I'm working?
While the queue is not yet committed I can change the commit log of a patch in a simple way.
hg qnew -f fix1 -m "this fixed issue 1" hg qnew -I debian/control -m "fix control" hg qnew -f fix1.1 -m "forgot something in issue 1" hg qgoto fix1 hg qfold fix1.1 # This concatenate the 2 comments" hg qrefresh -e # fix your comment as you like it hg qpush hg qfinish -a # commit all queues currently applied
Neato. This requires me to be in a queue *before* I fix my patch, right?
sandro *:-)
-- Sandro Dentella *:-) http://www.reteisi.org Soluzioni libere per le scuole http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
-- Jasper
On Tue, Jul 05, 2011 at 10:15:09PM -0400, Jasper St. Pierre wrote:
On Tue, Jul 5, 2011 at 5:02 AM, Alessandro Dentella <sandro@e-den.it> wrote:
On Tue, Jul 05, 2011 at 03:42:04AM -0400, Jasper St. Pierre wrote:
re: Mercurial, I didn't like it when I used it. If someone can tell me how to do this[3] in hg, I'd be more inclined to play along. And that
I do this sort of things using mercurial queues. I pile up patches in a queue and can subsequently navigate in the queue (hg qgoto fix_header1) and fold it with a later one (hg qfold fix_header2).
Hm. So it's like quilt?
I think so
Are patch queues real commits (changesets, revisions, whatever), so I can log and blame and grep them while I'm working?
yes for all 3 (log, blame, grep)
While the queue is not yet committed I can change the commit log of a patch in a simple way.
if the patch is called my_patch1: $ hg goto my_patch1 $ hg qrefresh -e (open editor to change edit log) hg qrefresh alone would just incorporate all modification to working directory in the patch, before committing you can anyhow keep all patches in a separate repository (hg qinit will initialize it for you. I personally don't use it though). If you use such a second repository I guess you can simply share that with other people too, but I'm not using this workflow.
hg qnew -f fix1 -m "this fixed issue 1" hg qnew -I debian/control -m "fix control" hg qnew -f fix1.1 -m "forgot something in issue 1" hg qgoto fix1 hg qfold fix1.1 # This concatenate the 2 comments" hg qrefresh -e # fix your comment as you like it hg qpush hg qfinish -a # commit all queues currently applied
Neato. This requires me to be in a queue *before* I fix my patch, right?
that's simpler. Otherwise you create a second patch and subsequently fold them toghether. Using a third part application called qct (that works also on git and some other I believe) you can also cherry pick single diffs in a single patch to be incorporated in a changeset. I use this a lot to keep the changeset as clean as possible. [disclaimer] I'm not an expert of git, so my comparison should not be taken seriously. It's true that all the time I use git I find it more convoluted than mercurial, and I always thought it was an historical heritage. sandro *:-) -- Sandro Dentella *:-) http://www.reteisi.org Soluzioni libere per le scuole http://sqlkit.argolinux.org SQLkit home page - PyGTK/python/sqlalchemy
Hi, just to note. If a move is preferred I give +1 for bitbucket (mercurial) If you ever want someone contributing under Windows, github with git is not a good solution. For Windows there are good clients for mercurial and bazzar. Git is more a Unix only solution. Launchpad has a horrible and unusable web ui so -1 on that. Also python has moved to mercurial and bitbucket catched up in features to github. Why should we move to a "no Python system" ? Regards, Wolfgang
On 5 Jul 2011, at 10:31, Wolfgang wrote:
If you ever want someone contributing under Windows, github with git is not a good solution. For Windows there are good clients for mercurial and bazzar. Git is more a Unix only solution.
I have no vote on the whole moving off SVN, but as a former windows user I'd like to make it clear that git has absolutely no issues with Windows and it has been so for 3 years now. Either in cygwin or by using the (officially linked from the git home page) msysgit standalone package, you get a completely functional git CLI tool plus a completely functional and awesome gitk graphical interface. I've been using that for a full year (including git-svn) and it's been working completely fine. (Of course it may have other limitations that I'm not aware of, but I haven't come across them). Finally, for what it's worth, for me as a potential contributor to Twisted (I still want to help with documentation) SVN is a much bigger barrier of entry than Trac. Even an official git mirror (complete with branches) that I could work against would be hugely beneficial. Git has a lot of local graphical tools that you can use to search, browse history, do diffs and so on, so that Trac+git can be a viable solution, even without Trac-git integration. Orestis
On Tue, Jul 5, 2011 at 11:05 AM, Orestis Markou <orestis@orestis.gr> wrote:
Finally, for what it's worth, for me as a potential contributor to Twisted (I still want to help with documentation) SVN is a much bigger barrier of entry than Trac. Even an official git mirror (complete with branches) that I could work against would be hugely beneficial. Git has a lot of local graphical tools that you can use to search, browse history, do diffs and so on, so that Trac+git can be a viable solution, even without Trac-git integration.
Orestis
Excellent! It looks like the Github *mirror* is going to be a thing, so that will at least make some of you happy. Unfortunately, it looks like the *move* (including tickets etc) to Github is never going to happen. I'm not going to elaborate. Someone else might. cheers lvh
On 07/05/2011 06:05 PM, Orestis Markou wrote:
On 5 Jul 2011, at 10:31, Wolfgang wrote:
If you ever want someone contributing under Windows, github with git is not a good solution. For Windows there are good clients for mercurial and bazzar. Git is more a Unix only solution.
I have no vote on the whole moving off SVN, but as a former windows user I'd like to make it clear that git has absolutely no issues with Windows and it has been so for 3 years now. Either in cygwin or by using the (officially linked from the git home page) msysgit standalone package, you get a completely functional git CLI tool plus a completely functional and awesome gitk graphical interface. I've been using that for a full year (including git-svn) and it's been working completely fine. (Of course it may have other limitations that I'm not aware of, but I haven't come across them).
Most people who stay on windows do not find cygwin or even CLI tools an acceptable solution. I think it is fair to say that git is a very unixy tool, and windows not its strong point when compared against bzr or hg (and I say that as someone who find git UI significantly better and easier then either bzr or hg). The situation on windows is consistantly improving though, with tools like smartgit which feel much more native to the typical windows user/developer. cheers, David
On Tue, Jul 5, 2011 at 12:22 PM, David <david@silveregg.co.jp> wrote:
Most people who stay on windows do not find cygwin or even CLI tools an acceptable solution.
So, the argument isn't that git is worse on Windows than it is on *nix: it's just that Windows users don't want to use CLI tools?
cheers, David
cheers lvh
On Tue, Jul 5, 2011 at 5:31 AM, Laurens Van Houtven <_@lvh.cc> wrote:
On Tue, Jul 5, 2011 at 12:22 PM, David <david@silveregg.co.jp> wrote:
Most people who stay on windows do not find cygwin or even CLI tools an acceptable solution.
So, the argument isn't that git is worse on Windows than it is on *nix: it's just that Windows users don't want to use CLI tools?
cheers, David
cheers lvh
Not in my opinion. I find hg, bzr, and svn all easier to use on Windows than git, and I use them all from the command line. For me the problem is that, while bash is certainly a superior "language" than Windows command language (a la cmd.exe), bash does not always map to Windows concepts/assumptions very nicely. This often leads to things occurring in unexpected (or at least unintuitive) ways (for Win32 users). Even though I use bash daily on Linux machines, I find using bash on Win32 painful (yes, even more painful than cmd.exe, which is really saying something!). Git requires bash. This makes it painful for me (on Windows). Also, Git _is_ worse on Windows than it is on *nix. It's just not as bad as it _used_ to be. It's functional. It works. But it is difficult to deal with, and a lot of Windows users I have talked to (as well as myself, of course) just don't like using it. I'm not necessarily saying that that means Twisted shouldn't use Git. But it _should_ be considered as a factor. Kevin Horn
On Tue, Jul 5, 2011 at 22:19, Kevin Horn <kevin.horn@gmail.com> wrote:
Git requires bash. This makes it painful for me (on Windows).
In what sense? You can run git from cmd.exe, without having to deal with bash. (You're not required to use 'Git Bash'.)
Also, Git _is_ worse on Windows than it is on *nix. It's just not as bad as it _used_ to be. It's functional. It works. But it is difficult to deal with, and a lot of Windows users I have talked to (as well as myself, of course) just don't like using it.
Is there anything in specific that is difficult? I haven't had Windows-specific problems with Git on Windows, and I've been using it a lot. Ivan
On Tue, Jul 5, 2011 at 5:26 PM, Ivan Kozik <ivan@ludios.org> wrote:
On Tue, Jul 5, 2011 at 22:19, Kevin Horn <kevin.horn@gmail.com> wrote:
Git requires bash. This makes it painful for me (on Windows).
In what sense? You can run git from cmd.exe, without having to deal with bash. (You're not required to use 'Git Bash'.)
Interesting. I was told (and had read) that it _was_ required. So I've been operating under that assumption. If you can run it in a cmd.exe window that _might_ relieve one of my pain points.
Also, Git _is_ worse on Windows than it is on *nix. It's just not as bad as it _used_ to be. It's functional. It works. But it is difficult to deal with, and a lot of Windows users I have talked to (as well as myself, of course) just don't like using it.
Is there anything in specific that is difficult? I haven't had Windows-specific problems with Git on Windows, and I've been using it a lot.
Ivan
Nothing terribly specific comes to mind, as I don't _use_ git very often. Only one of the projects I have ever contributed to uses git, and they just switched recently (from Mercurial, which makes very little sense to me as they are just about feature-equivalent). The others all use Mercurial, with the exception of Twisted. So when I started learning about DVCS, Mercurial was pretty much my introduction (aside: it seems to me that people in general seem to prefer whatever DVCS they were originally introduced to). Every couple of months I pull down a "new" release of git or TortoiseGit or whatever and tinker around with it, but it just isn't very nice compared to Mercurial. Maybe it will be someday. And it's not that anything is particularly _difficult_, so much as annoying. I find the CLI interface weird and clunky. I recall thinking some of the design decisions were not particularly good (though it's been long enough that I can't recall what they were exactly...and I have some complaints about the design of pretty much every DVCS out there, so...). These aren't Windows-specific issues of course, but when you add the Windows-specific issues on top of them, it just makes git that much worse to deal with. Of course, this is entirely subjective, and is totally my own opinion. Maybe the next time I update git it will annoy me less. As far as specific Windows-related issues, here's what I can come up with. These are all pretty vague, I'm afraid... - Let's start with installing. It would be really nice to be able to go to a website, download a package or archive or something, read some instructions and install git. Preferably with Tortoise-X-like Explorer shell integration (though I can live without that). I have never been able to do that. Instead, it's try the above, have it not work, search Google for a bunch of tutorial-style blog posts, try a bunch of stuff, maybe edit or move some files in whichever of several distributions I've had to download, and spend at least a couple of hours getting things working. At this point I'm already annoyed with git and I haven't even started using it yet. - Now I'm going to check things out. OK, fine. First hurdle is that terminology is different from what I'm used to, though that's hardly Git's fault, and I can deal with that. But in order to deal with the change in terminology, coming from Mercurial, I'd really like to see some nice online help. Last I checked, git totally failed in this area, and was noticeably worse on Windows than on *nix. (This was maybe 3 or 4 months ago). Ok, so now I'm having to search web/man pages for how to use git properly. Admittedly, some of this is necessary anyway for something as complex as a DVCS system, but it shouldn't be necessary for basic commands. I also seem to recall being annoyed by syntax for various commands actually being slightly _different_ on Windows than on *nix, but I can't say definitely that that was the case now (it was a long time ago, my memory is hazy, and I may have just misunderstood something). I have no idea if this is still a problem (assuming it ever really was). - I definitely miss Mercurial's friendly little warnings about when I might be about to screw things up. I doubt git will ever have anything like this. It seems antithetical to the git mindset. - Git just seems like it's second class citizen in the git world (this is true of a _huge_ number of open source projects, so it isn't just git, but I have a whole other rant about why this is a Bad Thing [tm] ). A lot of complaints about git on Win32 (which I find to be pretty valid complaints) I see answered with a sneer and something like: "Oh sure, but that's what you get for using _Windows_." To sum up, I use both Windows and Linux. That is unlikely to change any time soon. I want my tools to work nicely and be polished in both environments. Git has not impressed me as being able to do this yet. Even if it ever does get to this point, I still probably won't like it as much as Mercurial, just because hg is what I've become used to, and git doesn't offer much that hg doesn't already have. Well, this is already way longer than I had intended it to be... Kevin Horn
On 07/06/2011 07:19 AM, Kevin Horn wrote:
On Tue, Jul 5, 2011 at 5:31 AM, Laurens Van Houtven <_@lvh.cc> wrote:
On Tue, Jul 5, 2011 at 12:22 PM, David <david@silveregg.co.jp <mailto:david@silveregg.co.jp>> wrote:
Most people who stay on windows do not find cygwin or even CLI tools an acceptable solution.
So, the argument isn't that git is worse on Windows than it is on *nix: it's just that Windows users don't want to use CLI tools?
cheers, David
cheers lvh
Not in my opinion. I find hg, bzr, and svn all easier to use on Windows than git, and I use them all from the command line.
But I think you will think the same on unix, that is you will prefer hg/bzr to git on unix as well. I don't think someones will prefer hg over git on windows and prefer git over hg on unix, frankly. As for which is simpler, I think those differences are much more superficial than people want to think, and some concepts introduced in hg/bzr for simplicity sake actually harmful in the long term (e.g. natural revision number, especially as used in bzr). Git UI is not super consistent, but neither is hg as soon as you use e.g. named branches and bookmarks. cheers, David
On Wed, Jul 6, 2011 at 12:19 AM, Kevin Horn <kevin.horn@gmail.com> wrote:
Also, Git _is_ worse on Windows than it is on *nix. It's just not as bad as it _used_ to be. It's functional. It works. But it is difficult to deal with, and a lot of Windows users I have talked to (as well as myself, of course) just don't like using it.
I'm not necessarily saying that that means Twisted shouldn't use Git. But it _should_ be considered as a factor.
Kevin Horn
Gotcha, thanks. I've been told that hg is a lot more pleasant on Windows, (and you appear to echo that), and hg-git manages to be a damn-near 1:1 mapping. Have you tried that? cheers lvh
On Wed, Jul 6, 2011 at 5:35 AM, Laurens Van Houtven <_@lvh.cc> wrote:
On Wed, Jul 6, 2011 at 12:19 AM, Kevin Horn <kevin.horn@gmail.com> wrote:
Also, Git _is_ worse on Windows than it is on *nix. It's just not as bad as it _used_ to be. It's functional. It works. But it is difficult to deal with, and a lot of Windows users I have talked to (as well as myself, of course) just don't like using it.
I'm not necessarily saying that that means Twisted shouldn't use Git. But it _should_ be considered as a factor.
Kevin Horn
Gotcha, thanks.
I've been told that hg is a lot more pleasant on Windows, (and you appear to echo that), and hg-git manages to be a damn-near 1:1 mapping. Have you tried that?
cheers lvh
I've been told the same thing, by someone who _really_ didn't want to switch a project to git and got overruled. He indicated that using hg-git pretty much fixed all his complaints. I haven't used it myself, though if I ever need to do any serious work on a large project using git, I certainly intend to. Kevin Horn
Hi, I actually have contributed a few small patches to Twisted. I've attempted to contribute a few more which never made it in. I intend to contribute more in the future. I prefer launchpad over github because of its issue tracker, especially the ability to crosslink issues which affect more than one project or distribution. Also because the launchpad implementation is open source. But, I'll be happy to use whatever you folks want. Regards, Zooko
On Tue, Jul 5, 2011 at 9:31 AM, Wolfgang <tds333+tx@gmail.com> wrote:
Hi,
If you ever want someone contributing under Windows, github with git is not a good solution.
Why not? I know the reasons three years ago (and most of them were either permissions or performance), but I have been assured multiple times that this is no longer the case at all.
Also python has moved to mercurial and bitbucket catched up in features to github. Why should we move to a "no Python system" ?
Because the community on Github is significantly larger. At some point, this devolves into bikeshedding. Twisted devs would prefer Launchpad, but many people hate Launchpad with a passion. Between Github and Bitbucket, as you've said yourself, Bitbucket is playing feature catch-up (whether they're doing that successfully or not is something I'm willing to skip discussing). I don't think features are the thing to differentiate the two on (even though I think Github wins because of polish). It's network effects. Github has more following, so it's more interesting. The thing is, it's not so much a vote. This is a do-ocracy. The breaks were "I'll move it to Github, *or* someone stops me", not "I'll move it to Github, or Bitbucket, or whatever else you folks think is a good idea". If people are going to try and stop me from moving it to Github, I'm not going to move it to Bitbucket or anything else. It's just going to stay Trac/SVN. Regards,
Wolfgang
cheers lvh
On 5 July 2011 08:31, Wolfgang <tds333+tx@gmail.com> wrote:
just to note. If a move is preferred I give +1 for bitbucket (mercurial)
I thought it might be worth noting that CherryPy have recently migrated their website from Trac to Bitbucket * https://groups.google.com/group/cherrypy-users/browse_thread/thread/80fa8504... Perhaps in a few months they will have some information about how successful this has been. Sorry to bring back an old topic and I know there has been a lot of work done recently updating and improving the performance of the Twisted Trac site. -RichardW.
On 12:23 pm, m-lists@the-moon.net wrote:
On 5 July 2011 08:31, Wolfgang <tds333+tx@gmail.com> wrote:
just to note. If a move is preferred I give +1 for bitbucket (mercurial)
I thought it might be worth noting that CherryPy have recently migrated their website from Trac to Bitbucket
* https://groups.google.com/group/cherrypy- users/browse_thread/thread/80fa8504a2f247da?pli=1
Perhaps in a few months they will have some information about how successful this has been.
Sorry to bring back an old topic and I know there has been a lot of work done recently updating and improving the performance of the Twisted Trac site.
Improvements to the trac deployment shouldn't be seen as a reason not to consider entirely new solutions. Things can improve incrementally while plans are being made to improve them more drastically. Jean-Paul
participants (32)
-
Albert Brandl -
Alessandro Dentella -
anatoly techtonik -
Andrew Bennetts -
chris -
Christopher Armstrong -
David -
David Ripton -
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Ilja Livenson -
Itamar Turner-Trauring -
Ivan Kozik -
Jason J. W. Williams -
Jasper St. Pierre -
Johan Rydberg -
Jonathan Lange -
Kevin Horn -
Laurens Van Houtven -
Luke Marsden -
Michael Thompson -
Mikhail Terekhov -
Orestis Markou -
Phil Mayers -
Reza Lotun -
Richard Wall -
Thomas Hervé -
Tim Allen -
Tom Davis -
Tristan Seligmann -
Wolfgang -
Zooko O'Whielacronx