[Twisted-Python] Upcoming Twisted Release
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
---------
The plan is to push the release candidate only on PyPI and to host the release candidate documentation (including pydoctor generated API docs) only on Read the Docs site.
The Twisted Wiki Download page was updated to inform that PyPI is the only server where we publish the latest versions. So release candidates and final releases will be pushed only to PyPI. https://twistedmatrix.com/trac/wiki/Downloads
While the release candidate documentation in only published on Read The Docs, at least for the next release I plan to also publish the documentation to:
* https://twistedmatrix.com/documents/current/ * https://twistedmatrix.com/documents/VERSION/
After the release we can see if we can redirect them to Read The Docs.
----------------
For now, the only release blocker are the missing intersphinx API links on Read The Docs between the narrative docs and apidocs, but that is on track to be solved by next week.
So no other tickets are in the blocker queue: https://twistedmatrix.com/trac/report/26
------
Do you know any other release blocker issues?
Since we have good continuous testing, maybe we can stop doing release candidates.
The only thing blocking a release would be a "release blocket" ticket and truck CI checks not being green.
Cheers
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban adi@roiban.ro wrote:
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed? Unfortunately, Amber did not respond to any e-mails that I sent to her and Glyph, so I tried to move forward the best that I could.
So no other tickets are in the blocker queue: https://twistedmatrix.com/trac/report/26
Do you know any other release blocker issues?
I filed this: https://twistedmatrix.com/trac/ticket/10070
which I found with Pierre Tardy's help by running buildbot's test suite against Twisted trunk. This looks like a problem on the Twisted side, and should be fixed before a Twisted release is pushed out.
-- Craig
Hi Craig,
On Sun, 27 Dec 2020 at 20:10, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban adi@roiban.ro wrote:
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed?
If you want to do the release, I am more than happy to not have to do the release myself :)
Unfortunately, Amber did not respond to any e-mails that I sent to her and Glyph, so I tried to move forward the best that I could.
Is there anything still blocking you ? Can I help?
So no other tickets are in the blocker queue: https://twistedmatrix.com/trac/report/26
Do you know any other release blocker issues?
I filed this: https://twistedmatrix.com/trac/ticket/10070
which I found with Pierre Tardy's help by running buildbot's test suite against Twisted trunk. This looks like a problem on the Twisted side, and should be fixed before a Twisted release is pushed out.
OK. I closed https://twistedmatrix.com/trac/ticket/10069 as I think it's a duplicate.
Do you or Pierre plan to fix that ticket?
I think that you should block the release only if someone is committed to fixing the release blocker.
I plan to update the release documentation to make it clear that all release blocker tichet should have an owner and there are plans to fix the ticket in a maximum of 2 weeks.
Otherwise we risk to block the release forever... and if we delay forever people will start using "trunk" and if everybody is using trunk, what is the point of a release :) ?
------
For the future, maybe we can stop doing a release candidate wheel and instead just announce that a release is coming and people should test trunk and flag any release blocker ticket.
I think that with pip, is now very easy to install Twisted from trunk.
Cheers
--
Craig _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On 2020-12-27 18:58, Adi Roiban wrote:
For the future, maybe we can stop doing a release candidate wheel and instead just announce that a release is coming and people should test trunk and flag any release blocker ticket.
With all the work for automated releasing, isn't it easy to do an RC and a week or two later do a regular release? Admittedly, lots of people (like myself) haven't made most of our CI setups run all of pinned versions, latest release, latest pre-release, and latest trunk... but maybe having more projects actually doing release candidates is one piece of encouraging that? Or maybe, practically speaking, nobody is going this route anyways so you are suggesting a better path.
I think that with pip, is now very easy to install Twisted from trunk.
In case anyone isn't super familiar...
pip install git+https://github.com/twisted/twisted#egg=twisted
The #egg=twisted is useful for misc corner cases but not always required. One not-so-corner case is when you want to specify an extra then it is #egg=twisted[windows_platform], for example.
Cheers,
-kyle
On Sun, Dec 27, 2020 at 6:59 PM Adi Roiban adi@roiban.ro wrote:
Hi Craig,
On Sun, 27 Dec 2020 at 20:10, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban adi@roiban.ro wrote:
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed?
If you want to do the release, I am more than happy to not have to do the release myself :)
Unfortunately, Amber did not respond to any e-mails that I sent to her and Glyph, so I tried to move forward the best that I could.
Is there anything still blocking you ? Can I help?
So no other tickets are in the blocker queue: https://twistedmatrix.com/trac/report/26
Do you know any other release blocker issues?
I filed this: https://twistedmatrix.com/trac/ticket/10070
which I found with Pierre Tardy's help by running buildbot's test suite against Twisted trunk. This looks like a problem on the Twisted side, and should be fixed before a Twisted release is pushed out.
OK. I closed https://twistedmatrix.com/trac/ticket/10069 as I think it's a duplicate.
Do you or Pierre plan to fix that ticket?
I think that you should block the release only if someone is committed to fixing the release blocker.
I plan to update the release documentation to make it clear that all release blocker tichet should have an owner and there are plans to fix the ticket in a maximum of 2 weeks.
Otherwise we risk to block the release forever... and if we delay forever people will start using "trunk" and if everybody is using trunk, what is the point of a release :) ?
Part of the point is that when someone runs `pip install ...` they get a *working* version of Twisted, to the best of the project's ability to provide one.
Fortunately many regressions aren't that difficult to resolve. At worst, find the merge that introduced them and revert it. This works best when regressions are found in a timely manner, of course. Of course it's also nice if the problem can be fixed without backing out whatever (presumably desirable) set of changes it came along with.
Part of the release managers job is to motivate this kind of work to happen. A standing policy to revert the cause of a regression can also serve as good motivation to get the other kind of fix in, too.
It's better if these known regressions don't linger for months, though. It looks like the Buildbot PR had a failing CI run in October. I'd suggest that not waiting until December is a good way to avoid having these kinds of situations turn into a larger problem.
Jean-Paul
On Mon, 28 Dec 2020 at 01:00, Jean-Paul Calderone exarkun@twistedmatrix.com wrote:
On Sun, Dec 27, 2020 at 6:59 PM Adi Roiban adi@roiban.ro wrote:
Hi Craig,
On Sun, 27 Dec 2020 at 20:10, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban adi@roiban.ro wrote:
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed?
If you want to do the release, I am more than happy to not have to do the release myself :)
Unfortunately, Amber did not respond to any e-mails that I sent to her and Glyph, so I tried to move forward the best that I could.
Is there anything still blocking you ? Can I help?
So no other tickets are in the blocker queue: https://twistedmatrix.com/trac/report/26
Do you know any other release blocker issues?
I filed this: https://twistedmatrix.com/trac/ticket/10070
which I found with Pierre Tardy's help by running buildbot's test suite against Twisted trunk. This looks like a problem on the Twisted side, and should be fixed before a Twisted release is pushed out.
OK. I closed https://twistedmatrix.com/trac/ticket/10069 as I think it's a duplicate.
Do you or Pierre plan to fix that ticket?
I think that you should block the release only if someone is committed to fixing the release blocker.
I plan to update the release documentation to make it clear that all release blocker tichet should have an owner and there are plans to fix the ticket in a maximum of 2 weeks.
Otherwise we risk to block the release forever... and if we delay forever people will start using "trunk" and if everybody is using trunk, what is the point of a release :) ?
Part of the point is that when someone runs `pip install ...` they get a *working* version of Twisted, to the best of the project's ability to provide one.
Fortunately many regressions aren't that difficult to resolve. At worst, find the merge that introduced them and revert it. This works best when regressions are found in a timely manner, of course. Of course it's also nice if the problem can be fixed without backing out whatever (presumably desirable) set of changes it came along with.
Part of the release managers job is to motivate this kind of work to happen. A standing policy to revert the cause of a regression can also serve as good motivation to get the other kind of fix in, too.
It's better if these known regressions don't linger for months, though. It looks like the Buildbot PR had a failing CI run in October. I'd suggest that not waiting until December is a good way to avoid having these kinds of situations turn into a larger problem.
Jean-Paul
Thanks Kyle and Jean-Paul for your feedback.
With GitHub Actions is easy to have wheels and full docs published on PyPi and Read The Docs. We can do a release candidate and the release candidate might be a good way to do one more release rehearsal before the final release :)
I was wrong to suggest installing based on trunk ... even when use pip to install from git, it should have been installed based on the release branch :) But that issue is solved.
-----
I guess there are no comments against removing a ticket from the release-blocking list if the ticket is not active for 1 or 2 weeks.
-----
Crag, if you have time, can you join the IRC channel ?
Cheers
On Mon, Dec 28, 2020 at 6:13 AM Adi Roiban adi@roiban.ro wrote:
I plan to update the release documentation to make it clear that all release blocker tichet should have an owner and there are plans to fix the ticket in a maximum of 2 weeks.
Otherwise we risk to block the release forever... and if we delay forever people will start using "trunk" and if everybody is using trunk, what is the point of a release :) ?
Part of the point is that when someone runs `pip install ...` they get a *working* version of Twisted, to the best of the project's ability to provide one.
Fortunately many regressions aren't that difficult to resolve. At worst, find the merge that introduced them and revert it. This works best when regressions are found in a timely manner, of course. Of course it's also nice if the problem can be fixed without backing out whatever (presumably desirable) set of changes it came along with.
Part of the release managers job is to motivate this kind of work to happen. A standing policy to revert the cause of a regression can also serve as good motivation to get the other kind of fix in, too.
It's better if these known regressions don't linger for months, though. It looks like the Buildbot PR had a failing CI run in October. I'd suggest that not waiting until December is a good way to avoid having these kinds of situations turn into a larger problem.
Jean-Paul
Thanks Kyle and Jean-Paul for your feedback.
I guess there are no comments against removing a ticket from the release-blocking list if the ticket is not active for 1 or 2 weeks.
Commenting against this was the main reason for my earlier reply. I've left my quoted reply above.
Jean-Paul
On Dec 28, 2020, at 3:38 AM, Jean-Paul Calderone exarkun@twistedmatrix.com wrote:
I guess there are no comments against removing a ticket from the release-blocking list if the ticket is not active for 1 or 2 weeks.
Commenting against this was the main reason for my earlier reply. I've left my quoted reply above.
I'm also against this. Inactivity for a week or two is not enough reason to allow a known regression to be present in a release.
Is there a reason we can't identify the commit that caused the problem and simply revert before doing the release, then fix it afterwards?
If we're in a position where trunk has drifted so far between the regression and its identification that it is too labor-intensive to revert and we need to fix it forward, then the release is stuck - that's the whole point of the "release blocker" category. This is a very unfortunate situation but less unfortunate than shipping buggy releases that are known to break big users of Twisted.
-g
On Dec 28, 2020, at 9:50 AM, Glyph glyph@twistedmatrix.com wrote:
I'm also against this. Inactivity for a week or two is not enough reason to allow a known regression to be present in a release.
I should clarify that maybe I haven't processed the full context for this specific issue though. I do think it's acceptable to time out on an issue where a release blocker is claimed to exist if e.g. we can't reproduce the problem, or if it's a request for an above-and-beyond compatibility thing (like "please restore this private API, its removal breaks our application"). As a courtesy we might block a release for a small amount of time while waiting for a reproducer or a short-term private-API compatibility shim but the onus there is really on the reporter.
From what I can see though, this one is a pretty straightforward case of us just introducing a bug into a perfectly valid configuration though, just not one we happen to have in our test matrix right now.
-g
On Mon, Dec 28, 2020 at 10:04 AM Glyph glyph@twistedmatrix.com wrote:
From what I can see though, this one is a pretty straightforward case of us just introducing a bug into a perfectly valid configuration though, just not one we happen to have in our test matrix right now.
I am working on this right now, and am taking my time with the release. Since this is my first time doing a release, I am erring on the side of caution, which is why I am working to make sure that buildbot's tests pass, since they are a good representative project of a heavy Twisted user.
-- Craig
On Mon, 28 Dec 2020 at 18:17, Glyph glyph@twistedmatrix.com wrote:
On Dec 28, 2020, at 3:38 AM, Jean-Paul Calderone < exarkun@twistedmatrix.com> wrote:
I guess there are no comments against removing a ticket from the
release-blocking list if the ticket is not active for 1 or 2 weeks.
Commenting against this was the main reason for my earlier reply. I've left my quoted reply above.
I'm also against this. Inactivity for a week or two is not enough reason to allow a known regression to be present in a release.
Is there a reason we can't identify the commit that caused the problem and simply revert before doing the release, then fix it afterwards?
If we're in a position where trunk has drifted so far between the regression and its identification that it is too labor-intensive to revert and we need to fix it forward, then the release *is* stuck - that's the whole point of the "release blocker" category. This is a very unfortunate situation but less unfortunate than shipping buggy releases that are known to break big users of Twisted.
Thanks for your feedback.
I have created https://twistedmatrix.com/trac/ticket/10073 to add extra info to our release documentation.
So, we will continue to have release candidates and will block a release for as long as it's needed.
I can continue to use Twisted from pinned trunk without any issue :)
--------
It looks like with enough noise we got 2 PRs for the release blocker ticket
https://github.com/twisted/twisted/pull/1499 https://github.com/twisted/twisted/pull/1501
:)
So I hope the release will no longer be blocked for much longer :)
On Sun, Dec 27, 2020 at 3:59 PM Adi Roiban adi@roiban.ro wrote:
Hi Craig,
On Sun, 27 Dec 2020 at 20:10, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban adi@roiban.ro wrote:
Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro...
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed?
If you want to do the release, I am more than happy to not have to do the release myself :)
Well since Glyph gave me permission to do this, I would like to complete this.
OK. I closed https://twistedmatrix.com/trac/ticket/10069 as I think it's a duplicate.
Do you or Pierre plan to fix that ticket?
I'm working on this now.
It might take me a few days to complete this, since I am in the middle of holidays now, so won't be able to complete a release until the beginning of January.
-- Craig
On Dec 28, 2020, at 7:45 AM, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Sun, Dec 27, 2020 at 3:59 PM Adi Roiban <adi@roiban.ro mailto:adi@roiban.ro> wrote: Hi Craig,
On Sun, 27 Dec 2020 at 20:10, Craig Rodrigues <rodrigc@crodrigues.org mailto:rodrigc@crodrigues.org> wrote:
On Sat, Dec 26, 2020 at 3:50 PM Adi Roiban <adi@roiban.ro mailto:adi@roiban.ro> wrote: Hi,
I plan to act as a release manager for the next release and follow the plan documented at
https://docs.twistedmatrix.com/en/latest/core/development/policy/release-pro... https://docs.twistedmatrix.com/en/latest/core/development/policy/release-process.html
I was previously working on releasing Twisted. I was running into various roadblocks, but was moving forward, and got permission from Glyph to move forward with this. Has this changed?
If you want to do the release, I am more than happy to not have to do the release myself :)
Well since Glyph gave me permission to do this, I would like to complete this.
I'm not clear why Adi could not pick up your work where you left off?
OK. I closed https://twistedmatrix.com/trac/ticket/10069 https://twistedmatrix.com/trac/ticket/10069 as I think it's a duplicate.
Do you or Pierre plan to fix that ticket?
I'm working on this now.
It might take me a few days to complete this, since I am in the middle of holidays now, so won't be able to complete a release until the beginning of January.
Everybody should be able to take a break on a holiday (or heck, whenever you want, for as long as you want, we're all volunteers here) but "please don't make progress while I'm gone" is a surefire recipe for the person asking to complete the thing themselves getting distracted and wandering away. The release is stuck enough as it is with CI issues and regressions, so if someone has energy to make some forward progress let's please not hold it up.
-g
On Dec 28, 2020, at 9:53 AM, Glyph glyph@twistedmatrix.com wrote:
It might take me a few days to complete this, since I am in the middle of holidays now, so won't be able to complete a release until the beginning of January.
Everybody should be able to take a break on a holiday (or heck, whenever you want, for as long as you want, we're all volunteers here) but "please don't make progress while I'm gone" is a surefire recipe for the person asking to complete the thing themselves getting distracted and wandering away. The release is stuck enough as it is with CI issues and regressions, so if someone has energy to make some forward progress let's please not hold it up.
Following up on this, since we are now officially past the "beginning" of January: Craig, is a release imminent or can someone else pick up some tasks yet?
-g
On Fri, Jan 15, 2021 at 10:42 AM Glyph glyph@twistedmatrix.com wrote:
Following up on this, since we are now officially past the "beginning" of January: Craig, is a release imminent or can someone else pick up some tasks yet?
I'm close to getting this done, and want to complete this.
-- Craig
Hi,
So, speaking for myself here and not representing the Twisted community or anyone else.
I think that the consensus inside the Twisted contributors is that we should have a new release ASAP.
The current release is blocked by the following issues:
* A release blocker bug - https://twistedmatrix.com/trac/ticket/10069 * A release process protocol / procedure / communication
-------
For the blocker bug, there were 2 PR there were closed
* https://github.com/twisted/twisted/pull/1499 * https://github.com/twisted/twisted/pull/1501
There is one PR still in work https://github.com/twisted/twisted/pull/1502
We can wait one more week to see if this issue can be fixed and unblock the release in this way.
Otherwise, we can revert the change that introduced the regression and have the release unblocked.
Note that reverting a merge that introduced a regression as soon as the regression is detected, is the current best practice. The revert was not done as there was hope that this can be fixed.
With the release unblock Craig or I or someone else can create the release branch and push the release candidate to PYPI.
-------
For the release procedure / communication
The release documentation and the process to publish the wheels to PyPI was updated with the goal of making it easier for the current release to happen. You just create a tag and you have the documentation and PyPI wheels already published.
You can still manually create the wheels and publish them with twine
If anyone thinks that the current changes that were made for automatic Docs (narrative and API) and wheel publishing are a regression, they can be reverted at any time, and we can merge them after the release.
The ad-hoc / informal Twisted dev communication happens over IRC Freenode #twisted-dev
It would be nice if we can have all active developers there for quick feedback and brainstorming.
------------
Have I forgotten anything?
I hope that in 2 weeks we can have a release candidate and by the end of February a final release.
Cheers
On Sat, 16 Jan 2021 at 02:56, Craig Rodrigues rodrigc@crodrigues.org wrote:
On Fri, Jan 15, 2021 at 10:42 AM Glyph glyph@twistedmatrix.com wrote:
Following up on this, since we are now officially past the "beginning" of January: Craig, is a release imminent or can someone else pick up some tasks yet?
I'm close to getting this done, and want to complete this.
-- Craig _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Mon, Feb 1, 2021 at 11:27 AM Adi Roiban adi@roiban.ro wrote:
Hi,
So, speaking for myself here and not representing the Twisted community or anyone else.
Thanks for making this clear that you are speaking for yourself and no one else.
- A release blocker bug - https://twistedmatrix.com/trac/ticket/10069
For this particular bug, I have been working on getting feedback from Tom Most, under https://github.com/twisted/twisted/pull/1502 and am working on a fix to incorporate his feedback. This some central code is in base.py, and I am proceeding cautiously to make sure that I fully understand Tom's concern and am working on a fix.
I apologize if I have not been working fast enough on this release to your satisfaction.
I will attempt to communicate better on this list my progress.
I didn't want to bring this up on the mailing list. I will take your behavior as enthusiasm for the project, but despite the e-mail that Glyph sent, I still find your behavior very aggressive (and unappreciated). In your e-mail you are throwing out timelines, which may be "friendly suggestions" on your part, but can come across as deadlines/ultimatums.
For now, I am focusing on the final issue, and will proceed with the release as best as I am able, and communicate to the Twisted community my progress.
-- Craig
On Sun, Dec 27, 2020 at 3:59 PM Adi Roiban adi@roiban.ro wrote:
Is there anything still blocking you ? Can I help?
I do not need specific help from you at the moment, but when I do, I will ask. Thanks.
-- Craig
participants (5)
-
Adi Roiban -
Craig Rodrigues -
Glyph -
Jean-Paul Calderone -
Kyle Altendorf