Preparing the migration from Trac tickets to GitHub Issues
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, In preparation for the Trac ticket migration to GitHub Issues a staging/testing repository was created You can find the list of migrated issues here https://github.com/twisted/twisted-trac-migration/issues Send any feedback at the following ticket and my colleague Dan will try to address all the comments: https://github.com/twisted/twisted-trac-migration/issues/1 A bit of hard work is left to find the mapping between Trac users and GitHub users Handling at: https://github.com/twisted/twisted-trac-migration/issues/2 -------------- Once the migration starts, Trac will be in read-only mode with a message on the Trac wiki start page. The plan is to keep it only in read mode for a few more days until redirections are implemented. We still need to see how to put the GitHub twisted/twisted repo in a protected mode during the migration. One option to temporarily rename the main branch as "DONT-CREATE-A-PR-YET" and have a new PR template that will explain that people should wait 1 day for the migration to be finalized. Suggestions are welcomed for how to do the migration. Making the repo private is not an option. The followers/starts will be lost, and repo owners can still create PRs. ---------------- There is a list of issues with ID from 1 to 1732 So Trac ID 1 was migrated as https://github.com/twisted/twisted-trac-migration/issues/11323 Trac tickets with IDs 1733 or greater have a direct mapping. So Trac with ID 1733 is https://github.com/twisted/twisted-trac-migration/issues/1733 ---------- The migration will take about 1 day. This is slow as GitHub API is throttling the requests. The high-level plan targeting 4-5 June for going live: * Gather feedback in the next days and implement the required changes. * Next Monday, 30rd of May, re-run a full import for testing * By next Wednesday, the 1st of June, the testing import should be ready * Review the full migration and see if this is ready for going live. * Wait for feedback until Friday 3 of June. * If there are no complaints, the final / PROD migration will be triggered over the 4-5 June weekend. ----------------- Below are some examples. # Reopened, enhancement, link to another ticket https://github.com/twisted/twisted-trac-migration/issues/3621 # Reopened, attachment https://github.com/twisted/twisted-trac-migration/issues/4258 # a lot of changes and attachments https://github.com/twisted/twisted-trac-migration/issues/4536 # Enhancement, second comment replies to different ticket https://github.com/twisted/twisted-trac-migration/issues/6887 # Reopened -> new https://github.com/twisted/twisted-trac-migration/issues/9300 # Reopened -> closed, milestone, code example https://github.com/twisted/twisted-trac-migration/issues/9335 # With assigned team member, enhancement, new milestone, fixed https://github.com/twisted/twisted-trac-migration/issues/10027 Thanks -- Adi Roiban
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
Thank you for making this happen Adi! Sorry for the slow feedback, but here are a few comments. None seem blocking to me; it's totally xfine to proceed per the schedule. On Tue, May 24, 2022, at 4:56 AM, Adi Roiban wrote:
Perhaps set interaction limits for the duration of the migration as well? https://github.com/twisted/twisted/settings/interaction_limits
Could these go in a repo like https://github.com/twisted/trac so that they can be referenced under the original numbers as "trac#1"? I see that syntax used in some of the migrated tickets.
Could the branch name here be made a link? It corresponds to https://github.com/twisted/twisted/tree/password-comparison-4536-2 Not a big deal though.
The rest of these look great! Looking forward to much simpler infra. Thanks again, Tom
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, The migration will be postponed for the next weeks to address the current feedback. The plan is to address the current feedback and run another testing migration to make sure all is ok. Only after that, we will do the final (production) migration. The GitHub issues is the big migration, but along the way, we will also migrate the wiki. The Wiki will also be migrated, as I hope it will be easier to setup the redirections. Thanks, Tom for the feedback. I have updated the info from https://github.com/twisted/twisted-trac-migration/issues/1
Perhaps set interaction limits for the duration of the migration as well?
Yes. Good idea.
Could these go in a repo like https://github.com/twisted/trac so that they can be referenced under the original numbers as "trac#1"? I see that syntax used in some of the migrated tickets.I
I don't know. My idea is to stop hosting Trac on Azure VM. Instead, we will have a new website - like this https://twisted.org/, designed to be hosted on serverless environments - GitHub pages or Read The Docs or CloudFlare pages or Netlify...etc
Could the branch name here be made a link? It corresponds to https://github.com/twisted/twisted/tree/password-comparison-4536-2
The branch linking will be fixed...and I guess is already fixed. ------------- Glyph also sent feedback about the migration of the review queue. The main migration task is for https://twisted.reviews/ It can be replaced by a simple link, filtering "review" label, https://github.com/twisted/twisted-trac-migration/issues?q=is%3Aopen+is%3Ais... This needs to be coordinated with Glyph who manages that domain... or maybe look at hosting twisted.reviews on GitHub pages ------------- There are still some tasks blocking the migration. After the migration, we need to set up the redirections from Trac to GitHub Issues... to make sure people will not continue to use Trac Tickets There is this PR to migrate the whole Trac Wiki to static pages https://github.com/twisted/twisted.github.io/pull/10 Re On Fri, 3 Jun 2022 at 19:20, Tom Most <twm@freecog.net> wrote:
-- Adi Roiban
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, Migration is slow :) but there is progress :) A new testing migration was done here https://github.com/twisted/twisted-trac-migration-2/issues It should have the branch link fixed. Let me know if anything else should be changed. Otherwise, this is the content that will be migrated. Including the ticket ID conversion from Trac ID to GitHub ID for the Trac tickets that overlap with Github Pull IDs. In Github PR and Issues share the same ID space. ---------------- Besided the tickets we also need to migrate A. Trac Tickets reports This is-low importance. The plan is to replace them, after the migration, with some pre-defined github issues search URLs ----------- B. Static documentation pages Are the static files hosted at https://twistedmatrix.com/documents/ They contain the documentation pages and API documentation. For versions up to 14.2.0 we have the narrative docs on Read The Docs ... but not the API docs. https://docs.twistedmatrix.com/en/twisted-14.0.2/ The plan is to host them as static HTML pages in a Github repo, and server them over GitHub pages. Maybe over a twisted/twisted-docs-archive repo. Happy to receive suggestions for a better repo name and domain name for these pages. ---------- C Trac Wiki pages The plan is to store the existing wiki pages as normal files with edit revisions in a git repo. In this way, we preserve all the edit author and edit time and edit comments. The git repo will be read only. The long term plan is to stop using the wiki pages. Any valuable content that is not hosted on the wiki, should be moved into our narrative docs pages , hosted at https://docs.twistedmatrix.com/ ------------ The unsolved problem is how to put all these migrated pages together under a single domain. GitHub Pages has no support for redirect. We can create placeholder HTML pages with <meta http-equiv="refresh" content="0; url=$URL"> Another option is to have the https://twisted.org/ hosted on Netlify as a gateway and only implement it as a reverse proxy / redirection gateway. The main content will continue to be stored on GitHub pages. Any suggestions ? ------------- The plan is to put all this migration together under the https://twisted.org/ domain and website. After we are happy with the migration, we can just create twistedmatrix.com as an alias for the twisted.org domain. The goal is that with https://twisted.org/ can can experiment and test in real file the migration. My plan is to start first with the wiki pages. Part 1 - Create the archive repo. Part 2 - Host a "snapshot" for the latests wiki version as plain HTML files - https://github.com/twisted/twisted.github.io/pull/10 Part 3 - Make the current Trav wiki read-only ------------- Any questions or comments. let me know. On Sun, 5 Jun 2022 at 01:51, Glyph <glyph@twistedmatrix.com> wrote:
-- Adi Roiban
![](https://secure.gravatar.com/avatar/130405f65521b6ee6bbbe0204070a163.jpg?s=120&d=mm&r=g)
Hi Adi, Thanks for working on this -- taking a look at the issues that get generated I can see my comments coming across in some (e.g. https://github.com/twisted/twisted-trac-migration-2/issues/8928) but I'm surprised that I didn't end up on GitHub's subscribed list for the issue, even though they @clokep. Is this just a quirk of the import and will we need to search the meta-data / re-subscribe to any issues after the migration is complete? It looks like the metadata is easy enough to search at least! (https://github.com/twisted/twisted-trac-migration-2/issues?q=clokep seems to find all the things I should be CCed on or have reported, weirdly searching for cc_clokep, which is what is in the metadata, didn't find anything.) --Patrick On 6/16/22 9:35 AM, Adi Roiban wrote:
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi Patrick, On Thu, 16 Jun 2022 at 20:08, Patrick Cloke <patrick@cloke.us> wrote:
This is the limitation of the bulk import API. We can't impersonate other GitHub accounts, and can only take limited actions on behalf of the Twisted GitHub team members. We can't use the API to subscribe random accounts. You can only subscribe if you are watching the repo or if you explicitly left a comment on that GitHub issues. But all the comments so far are left by the import "twisted-trac" account.
You need to search for cc__clokep We were not able to use a single underscore or dash as GitHub has extra processing for these characters, and consider them word-breaks. See this link https://github.com/twisted/twisted-trac-migration-2/issues?q=is%3Aissue+is%3... Regards
-- Adi Roiban
![](https://secure.gravatar.com/avatar/130405f65521b6ee6bbbe0204070a163.jpg?s=120&d=mm&r=g)
On 6/17/22 5:23 AM, Adi Roiban wrote: that issue /after/ the import finished.
Oh I had missed that there was a double underscore there. Thanks! Looks like it will be easy enough to re-subscribe to things than afterward. Thanks so much for your explanation (and work on this)! Looking forward to using a single platform for Twisted. --Patrick
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, The latest plan is to wait this week for more feedback about the Trac to Github issues migration. If nobody asked for the migration to be blocked, the plan is to do the "real deal" next Monday, during UK working hours. ------------ About the wiki migration, more info is below. On Sat, 18 Jun 2022 at 00:17, Glyph <glyph@twistedmatrix.com> wrote:
Yes. The ticket migration is ongoing. the wiki migration is independent of the issue migration. While waiting for feedback for the ticket migration, I wanted to go ahead and do some work for the other parts of Trac and twistedmatrix.com website. The existing VM can stay online for as long as we want. Note that since it was created I haven't logged it to do any software or security updates. This is why I would like to put if offline ASAP. Also, instead of updating the internal VM configuration and the Twisted web application that is used to serve Trac, I would prefer to leave it as it is, and do the redirection via external services, as this is what we will use for the long term. Regards -- Adi Roiban
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Cool. Thanks, that was my main concern.
Love to see work happening in parallel without blocking :).
FWIW I do log in periodically and do that. And since you mentioned it I just rebooted it now ;-).
This is why I would like to put if offline ASAP.
It would definitely be nice to junk the whole thing. I still really want to set up some dynamic subdomains of twisted.org <http://twisted.org/> to do cool and interesting things with Twisted, and a green-field deployment of some freshly built containers would be much nicer than dragging this awkwardly-configured VM limping along.
+1, this all sounds good.
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
Thank you for making this happen Adi! Sorry for the slow feedback, but here are a few comments. None seem blocking to me; it's totally xfine to proceed per the schedule. On Tue, May 24, 2022, at 4:56 AM, Adi Roiban wrote:
Perhaps set interaction limits for the duration of the migration as well? https://github.com/twisted/twisted/settings/interaction_limits
Could these go in a repo like https://github.com/twisted/trac so that they can be referenced under the original numbers as "trac#1"? I see that syntax used in some of the migrated tickets.
Could the branch name here be made a link? It corresponds to https://github.com/twisted/twisted/tree/password-comparison-4536-2 Not a big deal though.
The rest of these look great! Looking forward to much simpler infra. Thanks again, Tom
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, The migration will be postponed for the next weeks to address the current feedback. The plan is to address the current feedback and run another testing migration to make sure all is ok. Only after that, we will do the final (production) migration. The GitHub issues is the big migration, but along the way, we will also migrate the wiki. The Wiki will also be migrated, as I hope it will be easier to setup the redirections. Thanks, Tom for the feedback. I have updated the info from https://github.com/twisted/twisted-trac-migration/issues/1
Perhaps set interaction limits for the duration of the migration as well?
Yes. Good idea.
Could these go in a repo like https://github.com/twisted/trac so that they can be referenced under the original numbers as "trac#1"? I see that syntax used in some of the migrated tickets.I
I don't know. My idea is to stop hosting Trac on Azure VM. Instead, we will have a new website - like this https://twisted.org/, designed to be hosted on serverless environments - GitHub pages or Read The Docs or CloudFlare pages or Netlify...etc
Could the branch name here be made a link? It corresponds to https://github.com/twisted/twisted/tree/password-comparison-4536-2
The branch linking will be fixed...and I guess is already fixed. ------------- Glyph also sent feedback about the migration of the review queue. The main migration task is for https://twisted.reviews/ It can be replaced by a simple link, filtering "review" label, https://github.com/twisted/twisted-trac-migration/issues?q=is%3Aopen+is%3Ais... This needs to be coordinated with Glyph who manages that domain... or maybe look at hosting twisted.reviews on GitHub pages ------------- There are still some tasks blocking the migration. After the migration, we need to set up the redirections from Trac to GitHub Issues... to make sure people will not continue to use Trac Tickets There is this PR to migrate the whole Trac Wiki to static pages https://github.com/twisted/twisted.github.io/pull/10 Re On Fri, 3 Jun 2022 at 19:20, Tom Most <twm@freecog.net> wrote:
-- Adi Roiban
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, Migration is slow :) but there is progress :) A new testing migration was done here https://github.com/twisted/twisted-trac-migration-2/issues It should have the branch link fixed. Let me know if anything else should be changed. Otherwise, this is the content that will be migrated. Including the ticket ID conversion from Trac ID to GitHub ID for the Trac tickets that overlap with Github Pull IDs. In Github PR and Issues share the same ID space. ---------------- Besided the tickets we also need to migrate A. Trac Tickets reports This is-low importance. The plan is to replace them, after the migration, with some pre-defined github issues search URLs ----------- B. Static documentation pages Are the static files hosted at https://twistedmatrix.com/documents/ They contain the documentation pages and API documentation. For versions up to 14.2.0 we have the narrative docs on Read The Docs ... but not the API docs. https://docs.twistedmatrix.com/en/twisted-14.0.2/ The plan is to host them as static HTML pages in a Github repo, and server them over GitHub pages. Maybe over a twisted/twisted-docs-archive repo. Happy to receive suggestions for a better repo name and domain name for these pages. ---------- C Trac Wiki pages The plan is to store the existing wiki pages as normal files with edit revisions in a git repo. In this way, we preserve all the edit author and edit time and edit comments. The git repo will be read only. The long term plan is to stop using the wiki pages. Any valuable content that is not hosted on the wiki, should be moved into our narrative docs pages , hosted at https://docs.twistedmatrix.com/ ------------ The unsolved problem is how to put all these migrated pages together under a single domain. GitHub Pages has no support for redirect. We can create placeholder HTML pages with <meta http-equiv="refresh" content="0; url=$URL"> Another option is to have the https://twisted.org/ hosted on Netlify as a gateway and only implement it as a reverse proxy / redirection gateway. The main content will continue to be stored on GitHub pages. Any suggestions ? ------------- The plan is to put all this migration together under the https://twisted.org/ domain and website. After we are happy with the migration, we can just create twistedmatrix.com as an alias for the twisted.org domain. The goal is that with https://twisted.org/ can can experiment and test in real file the migration. My plan is to start first with the wiki pages. Part 1 - Create the archive repo. Part 2 - Host a "snapshot" for the latests wiki version as plain HTML files - https://github.com/twisted/twisted.github.io/pull/10 Part 3 - Make the current Trav wiki read-only ------------- Any questions or comments. let me know. On Sun, 5 Jun 2022 at 01:51, Glyph <glyph@twistedmatrix.com> wrote:
-- Adi Roiban
![](https://secure.gravatar.com/avatar/130405f65521b6ee6bbbe0204070a163.jpg?s=120&d=mm&r=g)
Hi Adi, Thanks for working on this -- taking a look at the issues that get generated I can see my comments coming across in some (e.g. https://github.com/twisted/twisted-trac-migration-2/issues/8928) but I'm surprised that I didn't end up on GitHub's subscribed list for the issue, even though they @clokep. Is this just a quirk of the import and will we need to search the meta-data / re-subscribe to any issues after the migration is complete? It looks like the metadata is easy enough to search at least! (https://github.com/twisted/twisted-trac-migration-2/issues?q=clokep seems to find all the things I should be CCed on or have reported, weirdly searching for cc_clokep, which is what is in the metadata, didn't find anything.) --Patrick On 6/16/22 9:35 AM, Adi Roiban wrote:
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi Patrick, On Thu, 16 Jun 2022 at 20:08, Patrick Cloke <patrick@cloke.us> wrote:
This is the limitation of the bulk import API. We can't impersonate other GitHub accounts, and can only take limited actions on behalf of the Twisted GitHub team members. We can't use the API to subscribe random accounts. You can only subscribe if you are watching the repo or if you explicitly left a comment on that GitHub issues. But all the comments so far are left by the import "twisted-trac" account.
You need to search for cc__clokep We were not able to use a single underscore or dash as GitHub has extra processing for these characters, and consider them word-breaks. See this link https://github.com/twisted/twisted-trac-migration-2/issues?q=is%3Aissue+is%3... Regards
-- Adi Roiban
![](https://secure.gravatar.com/avatar/130405f65521b6ee6bbbe0204070a163.jpg?s=120&d=mm&r=g)
On 6/17/22 5:23 AM, Adi Roiban wrote: that issue /after/ the import finished.
Oh I had missed that there was a double underscore there. Thanks! Looks like it will be easy enough to re-subscribe to things than afterward. Thanks so much for your explanation (and work on this)! Looking forward to using a single platform for Twisted. --Patrick
![](https://secure.gravatar.com/avatar/eba6eb871de2549c7447a8701352cd35.jpg?s=120&d=mm&r=g)
Hi, The latest plan is to wait this week for more feedback about the Trac to Github issues migration. If nobody asked for the migration to be blocked, the plan is to do the "real deal" next Monday, during UK working hours. ------------ About the wiki migration, more info is below. On Sat, 18 Jun 2022 at 00:17, Glyph <glyph@twistedmatrix.com> wrote:
Yes. The ticket migration is ongoing. the wiki migration is independent of the issue migration. While waiting for feedback for the ticket migration, I wanted to go ahead and do some work for the other parts of Trac and twistedmatrix.com website. The existing VM can stay online for as long as we want. Note that since it was created I haven't logged it to do any software or security updates. This is why I would like to put if offline ASAP. Also, instead of updating the internal VM configuration and the Twisted web application that is used to serve Trac, I would prefer to leave it as it is, and do the redirection via external services, as this is what we will use for the long term. Regards -- Adi Roiban
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Cool. Thanks, that was my main concern.
Love to see work happening in parallel without blocking :).
FWIW I do log in periodically and do that. And since you mentioned it I just rebooted it now ;-).
This is why I would like to put if offline ASAP.
It would definitely be nice to junk the whole thing. I still really want to set up some dynamic subdomains of twisted.org <http://twisted.org/> to do cool and interesting things with Twisted, and a green-field deployment of some freshly built containers would be much nicer than dragging this awkwardly-configured VM limping along.
+1, this all sounds good.
participants (4)
-
Adi Roiban
-
Glyph
-
Patrick Cloke
-
Tom Most