On 12/03/2014 23:52, Glyph Lefkowitz wrote:

On Mar 12, 2014, at 3:43 PM, Christopher Armstrong <radix@twistedmatrix.com <mailto:radix@twistedmatrix.com>> wrote:

...

On March 12, 2014 at 5:39:36 PM, Chris Withers (chris@simplistix.co.uk <mailto:chris@simplistix.co.uk>) wrote:

...

Hi All,

Is there a git mirror of the twisted repo anywhere?

Yes, it’s at

http://github.com/twisted/twisted

Also at https://code.twistedmatrix.com/git/Twisted.

However, if you actually want to /use/ this mirror to work on Twisted, <https://twistedmatrix.com/trac/wiki/GitMirror> has some useful information.

Not to be too contentious, but when do you reckon you guys will switch the main repo to git? The difficulty of using svn for the main repo is that it only has a subset of the features of a dvcs such as git. The difficulty of using git for the main repo is if you have people who still want to interact via svn. Well, it would be if you didn't use github: https://help.github.com/articles/support-for-subversion-clients I do wonder how many people know about the above feature, it blows my mind a bit that it's actually there in the first place... Anyway, just a thought :-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On 03/13/2014 03:14 AM, HawkOwl wrote:

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Hi Chris,

I can assure you this topic has come up many times - and unless we have volunteers that can manage to get the several thousand tickets with comment history into GitHub (which I'm sure that nobody wants to get tied to when you have a workflow already), without losing anything, then it's most likely Twisted is sticking with svn and trac.

Git has svn support - see `git svn`. I have merged several branches with it, and it works great. Use that instead, if you get commit access :)

Personally I feel like these are two separate things - we could switch to Git while worrying about tickets later. Still would take some infrastructure work, but a lot less. -Itamar

Hi, On Thu, Mar 13, 2014 at 12:04 PM, Jonathan Ballet <jon@multani.info> wrote:

Why does using Git involve moving tickets over to Github? Why does using Git even involve Github at all, in the first place?

HawkOwl probably meant that it would be hard to get all the tooling (trac, ci...) working while switching up the VCS out from under it. There's a *lot* of stuff that relies on SVN details. Plus, with the git mirrors, we don't have to do that work, and anyone who only wants to work in git all day, can. hth lvh

Hi Glyph, On Thu, Mar 13, 2014 at 12:35:17PM -0700, Glyph Lefkowitz wrote:

On Mar 13, 2014, at 4:14 AM, Laurens Van Houtven <_@lvh.io> wrote:

...

HawkOwl probably meant that it would be hard to get all the tooling (trac, ci...) working while switching up the VCS out from under it. There's a *lot* of stuff that relies on SVN details.

I think handwaving over the list of things that need to be done might make people unnecessarily afraid to work on this. The list is finite :-).

All of our infrastructure is (or at least, should be) available here <https://github.com/twisted-infra/braid>. It should be possible to submit a PR against that repository (and the relevant submodules) which makes the update and then just deploy it with 'fab config.production'.

The major blocker is that we need to upgrade Trac to get a version of Trac that supports a Git source browser reasonably, and can detect ticket references and ticket closures from Git commit messages. This is currently stuck: <https://twistedmatrix.com/trac/wiki/Infrastructure/UpgradingTrac>. This would be good for lots of other reasons, but upgrading trac has proven very difficult in the past.

We might also need to install a trac plugin of some kind for the git stuff, I'm not sure about that.

We would also need to port the post-commit hooks in SVN which tell Kenaan to announce commits on IRC, enforce repository structure (i.e. no merges to trunk, or, presumably, "master", without a fixed bug and a NEWS file, unless it's a quote file change), and kick off buildbot.

Right now there's some SVN mirroring stuff to keep SVN updated on Trac, since we don't want our release repository on the same machine running the website. (code.twistedmatrix.com has a much smaller attack surface).

The CI system actually already uses git, in preparation for this merge, so I don't think there's a whole lot of work there. We might not even need to set a different upstream URL.

If someone would like to turn this list into a wiki page under <https://twistedmatrix.com/trac/wiki/Infrastructure> that would be great, so it could be updated with anything I've forgotten. Right now there's the beginnings of a SwitchToBazaar which should probably be deleted, since we're not going to do that.

I don't have edit permissions on the wiki, so I took the list of stuff you mentionned here and made a wiki page joined to this email. Compared to the SwitchToBazar page, here are the things which are missing from your email - I don't know how relevant they are though: * mailer post-commit hook * the release scripts * the developer documentation (I guess that should be done) * figure the version number from Git instead of Subversion * decide what to do with Combinator There might be still some editing to do but that could be a start. Jonathan

On Sun, Mar 16, 2014 at 01:12:42PM -0700, Glyph wrote:

...

...

If someone would like to turn this list into a wiki page under <https://twistedmatrix.com/trac/wiki/Infrastructure> that would be great, so it could be updated with anything I've forgotten. Right now there's the beginnings of a SwitchToBazaar which should probably be deleted, since we're not going to do that.

I don't have edit permissions on the wiki, so I took the list of stuff you mentionned here and made a wiki page joined to this email.

What's your username? The only reason we even have edit permissions is to prevent spamming, anyone may ask for edit permission at any time :-).

Oh yeah, maybe that's needed... My username is 'multani', thanks for editing me!

...

Compared to the SwitchToBazar page, here are the things which are missing from your email - I don't know how relevant they are though:

* mailer post-commit hook

Yeah, we probably need this. It seems like this ought to be pretty trivial to set up in git though :-).

...

* the release scripts

Oh, good point. These rely fairly lightly on svn, but the dependency is there.

This would be a good place to start. Unfortunately, since tagging works differently in svn and git, I don't think we can simply start doing the release out of git without a repository to push to. If I'm wrong about this, so much the better, and we could actually start having release managers use git if they like :). Regardless, all the code to do so could be tested and landed as a normal development task with no administrative blockers, so this might be a good place for someone interested in moving the process along to start.

That's twisted.python._release right? greping for 'svn' in this module shows only 4 lines which are running the 'svn' command, for which only the 'rm' will actually change the repository (used to remove the NEWS fragments). I've created #7035 for this, I'll try to have a look today. When I'll have the edit permissions on the wiki, I'll update the SwitchToGit page to reference to this ticket.

...

* figure the version number from Git instead of Subversion

This would be an even more straightforward patch to Twisted, in twisted.python.versions.

I've created #7036 for this task.

...

* decide what to do with Combinator

I think Combinator is an evolutionary dead end at this point. Tools like virtualenv, virtualenvwrapper, pyenv,

It seems a part is missing here :) Jonathan

On Mon, Mar 17, 2014 at 01:34:28PM +0900, Jonathan Ballet wrote:

...

...

This would be an even more straightforward patch to Twisted, in twisted.python.versions.

I've created #7036 for this task.

So, I was looking at twisted.python.versions and I'm wondering how is it supposed to work with recent versions of Subversion (the ones which removed all the ".svn" in all the sub-directories, I don't know which one started this.)

Looking at https://twistedmatrix.com/trac/browser/trunk/twisted/python/versions.py#L210 (line 210 to 215), I don't know how it supposed to work with recent version of Subversion:

* the "twisted" module in this case would refer to the file "trunk/twisted/__init__.py" * but the ".svn" directory is going to be in "trunk/.svn" so the function always returns None

Actually, it's going to be the same thing with Git, the ".git" is one level higher than the package which is passed to Version() in this case. Version() can still run "git describe" or the like to get the latest changeset number but: * it's probably going to launch the "git" command which might be costly * if for some reason the package is not from a Git repository, but there's one (potentially completely different) in a parent directory, "git describe" will happily return the changeset number from this upper directory and it's going to give some funky revision number.

Additionally, it doesn't seem there's any parser for recent Subversion's database? So, how useful is that module at that moment?

Jonathan

On Sun, Mar 16, 2014 at 9:12 PM, Glyph <glyph@twistedmatrix.com> wrote:

On Mar 16, 2014, at 12:31 AM, Jonathan Ballet <jon@multani.info> wrote:

Hi Glyph,

On Thu, Mar 13, 2014 at 12:35:17PM -0700, Glyph Lefkowitz wrote:

If someone would like to turn this list into a wiki page under < https://twistedmatrix.com/trac/wiki/Infrastructure> that would be great, so it could be updated with anything I've forgotten. Right now there's the beginnings of a SwitchToBazaar which should probably be deleted, since we're not going to do that.

I don't have edit permissions on the wiki, so I took the list of stuff you mentionned here and made a wiki page joined to this email.

What's your username? The only reason we even have edit permissions is to prevent spamming, anyone may ask for edit permission at any time :-).

I would like edit permissions too, my username is brunsgaard.

Compared to the SwitchToBazar page, here are the things which are missing from your email - I don't know how relevant they are though:

* mailer post-commit hook

Yeah, we probably need this. It seems like this ought to be pretty trivial to set up in git though :-).

* the release scripts

Oh, good point. These rely fairly lightly on svn, but the dependency is there.

This would be a good place to start. Unfortunately, since tagging works differently in svn and git, I don't think we can simply start doing the release out of git without a repository to push to. If I'm wrong about this, so much the better, and we could actually start having release managers use git if they like :). Regardless, all the code to do so could be tested and landed as a normal development task with no administrative blockers, so this might be a good place for someone interested in moving the process along to start.

* the developer documentation (I guess that should be done)

The developer documentation is already so bad (as Adi Roiban has pointed out) that it wouldn't make sense to block migrating to git on a comprehensive upgrade.

* figure the version number from Git instead of Subversion

This would be an even more straightforward patch to Twisted, in twisted.python.versions.

* decide what to do with Combinator

I think Combinator is an evolutionary dead end at this point. Tools like virtualenv, virtualenvwrapper, pyenv,

There might be still some editing to do but that could be a start.

I've deleted the SwitchToBazaar page and, for now, uploaded your proposal at <https://twistedmatrix.com/trac/wiki/Infrastructure/SwitchToGit>. Hopefully someone (maybe me) can give you edit permission as soon as you announce your username :).

Thanks for your contribution,

-glyph

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

On 13/03/2014 11:17, HawkOwl wrote:

If you want git access, we have that already (both read and write!),\

So, if I hypothetically put in a pull request via GitHub, that would "just work"? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Mar 13, 2014, at 4:17 AM, HawkOwl <hawkowl@atleastfornow.net> wrote:

There is no other reason to do a base repo switch other than "svn is scary" or "I don't like svn".

There's a very good reason to do a base repo switch. Right now, every time we do a merge to trunk, we're losing all the metadata about version history; the individual commits within the branch don't show up in 'svn blame' so it's hard to determine how an individual line got to be the way it is. I'd very much like us to switch to git entirely so we can start preserving all of this information. I don't know if we'll use git forever, but the sooner we switch, the more history will be preserved for future version control systems (whatever we migrate to once the git fad has run its course ;-)). -glyph

On 13/03/2014 19:38, Glyph Lefkowitz wrote:

On Mar 12, 2014, at 11:53 PM, Chris Withers <chris@simplistix.co.uk <mailto:chris@simplistix.co.uk>> wrote:

...

Not to be too contentious, but when do you reckon you guys will switch the main repo to git?

Just to be clear about the nature of my other answers: we do not "reckon" such a thing will happen; we have no estimates. Things get done because volunteers do them. You could be just such a volunteer :).

I've read up on the thread and it's interesting to note that Mike Bayer has poured a huge amount of effort into doing the move for SQLAlchemy, producing some great tools in the process. I believe he now has 3 git repos that keep each other in sync, as well as accepting pull requests via both bitbucket and github. I asked him about it and checked that it was okay to forward on his reply, please find it attached. Hope this helps, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On March 18, 2014 at 6:38:42 PM, Jonas Brunsgaard (jonas.brunsgaard@gmail.com) wrote: I think that it is a shame that Twisted is not on Github(Yep Github, not just git) If we make a comparison between SVN and Git, I think it is obvious that Git is the superior VCS. That said, I think Github  - as a platform - creates a certain "feeling" of openness, transparency, accessibility and recognisability. All things, that make new users of twisted feel better about the project. As an example I think that most developers know how to submit PR's via Github, where I (as an example) don`t know how to submit a patch to Twisted today(of cause I could read up on it, but i already know the other flow). And that could hold back some people from submitting bugfixes etc. I think accepting PRs via Github would be a huge benefit to Twisted. I feel many developers see Twisted as good and stable, but also a little 'dusty' and old, and I think that a move to Github would bring more code submitters, more discussions, more accessibility and more ' generel development' in the project. For that reasons I feels like it may be a good idea to move twisted to Github. If at some point the core dev team decides that such a move should happen, I would love to help with the movement of issues etc.(all the slave work :)) (and I think others outside the core team would be willing to help out too)  While I doubt Twisted would move to Github’s pretty poor issue tracker, there is already some agreement that Twisted should be moved to github inasmuch that PRs should reasonably be integrated into the workflow. There are already some tickets on the Twisted tracker about a move to Git. I’m sure your help would be greatly appreciated :-) Here’s a wiki page where some of the tickets are linked: https://twistedmatrix.com/trac/wiki/Infrastructure/SwitchToGit --  Christopher Armstrong http://twitter.com/radix http://wordeology.com/

On 19 March 2014 01:51, Christopher Armstrong <radix@twistedmatrix.com> wrote: [snip]

That said, I think Github - as a platform - creates a certain "feeling" of openness, transparency, accessibility and recognisability.

Well.. it depends! GitHub is closed source and for any change you are at the mercy of GitHub developers or rely on GitHub API + external hacks to do what you want. This does not mean that Twisted can not be moved to GitHub... but I am not sure about the open part. GitHub ticket management is mediocre but I don't see any advanced ticket handling in Twisted's Trac either. <snip>

I think accepting PRs via Github would be a huge benefit to Twisted.

I feel many developers see Twisted as good and stable, but also a little 'dusty' and old, and I think that a move to Github would bring more code submitters, more discussions, more accessibility and more ' generel development' in the project.

For that reasons I feels like it may be a good idea to move twisted to Github.

If at some point the core dev team decides that such a move should happen, I would love to help with the movement of issues etc.(all the slave work :)) (and I think others outside the core team would be willing to help out too)

Me too. Happy to do the slave work! For my project, I already do something similar. I have a git *mirror* on GitHub, tickets in Trac and reviews over GitHub. Using Trac-API and GitHub-API I keep trac ticket in sync with GitHub pull request activity. As a non comitter sending patches and now reviewing patches is not fun... and I think that contributing to open source project should be fun.

While I doubt Twisted would move to Github’s pretty poor issue tracker, there is already some agreement that Twisted should be moved to github inasmuch that PRs should reasonably be integrated into the workflow.

Even if Twisted would completely move to GitHub, I think is very hard to do this move all at once. This is why, for me, it makes much sense to migrate review process, tickets, wiki (if required), website as independent steps. GitHub API and hooks are great and can help keep things in sync while the transition is done. I have not done any work in helping move reviews to GitHub since the CONTRIBUTING file contains a bold warning **Warning: pull requests are ignored!** This read to me like : Hey you fad driven developer! Stop using this fancy new, git, GitHub and PR things and go user our old good Trac and SVN tools. This is not a complain, just well intention feedback! Looking forward for working with Pull Requests in Twisted :) -- Adi Roiban