[Twisted-Python] The Python 2.7 Plan
![](https://secure.gravatar.com/avatar/c312623edb9666f92c17bf2c925947f4.jpg?s=120&d=mm&r=g)
With Python 2.7 support dropping in a matter of weeks, I'd like to pick up from hawkowl's thread back in March about Python 2.7 support for Twisted. hawkowl had shared a plan in a Google Docs document. That URL has since gone stale but here's a new one (thanks, hawkowl): https://docs.google.com/document/d/1lIkg6mYVMxVhh3_U6uSJ1XjllPM92Nf8Zp7b7OZP... Here's what I got from the prior conversation: Amber Brown (hawkowl):
As an example here, I'd like us to start adopting type hints. I find them incredibly valuable as documentation and for catching lots of bugs. The best way to adopt them is to use inline syntax supported by Python 3.6+, or slightly less ideal, Python 3.5+. It looks like pydoctor may have support for this soon, which makes that an increasingly interesting avenue. To support Python 2.7, we'd have to use either: - the comments-based inline syntax, which is far uglier and leaves us with an extra project of converting those later when we finally do drop 2.7, or - stub files, which are considerably less easy to manage over time Doing this will also afford us an opportunity to better express the API (do we mean to return a "mutable sequence" or a specifically a list, etc.). There's a longer conversation to be had about type hints, and that should be on a separate thread; I only talk about it here to explain why Python 2.7 is in the way.
Can you share how you looked up these stats? I don't see a way to do so with pypistats. John Santos:
To be clear, I would not expect this to be the case if you update Twisted to a version that no longer supports Python 2.7. You would have to continue using a version that does support Python 2.7. You should have some runway, however, in which your app still works with either Py2.7 and Twisted 19 as well as Py3.8 using Twisted 20, for example. Glyph, in reference to a maintenance branch for Python 2.7:
Personally, I'm not willing to commit to this. I know from experience both on Twisted and other projects that maintaining multiple release branches, even one that's "maintenance only", requires at least one point-person for each branch at all times (usually a "release manager"). And backporting fixes inevitably gets harder and harder as the "maintenance" branch diverges from top-of-tree. If I have time to work on Twisted in my increasingly scarce spare time, I want it to be on something at least plausibly interesting, and manually backporting manual fixes to an unmaintained py2 branch that I don't even use doesn't qualify.
I tend to agree. I think the compromise would be that we could be willing to accept back-ports of certain fixes (presumably limited to security fixes?) to a maintenance branch, but not that we would commit to doing so for any period of time. Additionally, we've already supported Python 2.7 for the entirely supported lifetime of Python 2.7. Glyph also notes some infrastructure we use that is blocked on code fixes: - Trac: https://trac.edgewall.org/ticket/12130 Looks like we can leave the Trac instance running in a Twisted 19 WCGI server until they fix that. - DNS: https://twistedmatrix.com/trac/ticket/9496 I'd suggest we do the same as with Trac until this is fixed, but Glyph disagrees. Back to hawkowl's document: - Part 1: I think we should drop the Py2.7 maintenance branch piece per Glyph's comment above. - Part 2: Re-introduced modules should go into separate repositories, not back into Twisted proper. As to the timeline, Option 1 would be to cut a 2.7 release in a couple of weeks. I don't know that we need to do this in January, but I would suggest that our next release is the final release for 2.7. This would require a fix to the DNS issue #9496. -wsv
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On Wed, Dec 11, 2019 at 4:06 PM Wilfredo Sánchez Vega <wsanchez@wsanchez.net> wrote:
With Python 2.7 support dropping in a matter of weeks, I'd like to pick up from hawkowl's thread back in March about Python 2.7 support for Twisted.
Despite the widely circulated January 1 2020 date, it seems that python-dev nevertheless plans to make a Python 2.7 release in April 2020 - https://www.python.org/dev/peps/pep-0373/ This is not to say the questions here don't need answering or that there isn't some urgency at this point, but it seems like the January 1 date is essentially an arbitrary deadline without a clear connection to the support status of CPython 2.7. Or does anyone know a different way to reconcile these apparently conflicting dates? Jean-Paul
![](https://secure.gravatar.com/avatar/c312623edb9666f92c17bf2c925947f4.jpg?s=120&d=mm&r=g)
I don't object to the final release being in April instead of January, though I'm not sure that the additional release really means we need to do that. But FWIW I'm in favor of Option 3 / calling 19.4 the final release. :-) My sense is that there was consensus on hawkowl's proposal except for, I think: 1- Should we drop the idea of a 2.7 maintenance branch? 2- It seems we need to fix the bug in twisted.names first. 3- Choosing a final 2.7 release date. -wsv
![](https://secure.gravatar.com/avatar/3d37232726396a1d3c7412dd915095ea.jpg?s=120&d=mm&r=g)
On 12/12/19 8:41 am, Wilfredo Sánchez Vega wrote:
JP has mentioned elsewhere that enough of the ecosystem has dropped, or will drop, support that a lot of applications staying on 2.7 will have dependencies frozen in time regardless. I think this is valid, and that Twisted remaining compatible with 2.7 hardly matters when so much else isn't and forces the hand of 2.7-stalwarts regardless. As such, I think dropping the 2.7 maint branch is valid. And yes, we do need to fix that bug. There's some other py3 specific bugs in names's host file resolver, but, I think that bugfix would end up hitting that portion of code's unicode uncleanliness. It just comes down to someone doing it, I guess... - Amber
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
This is just speculative, just based on my own historical understanding of python-dev's process and somewhat informal definition of "support", but I believe the sentence "Support officially stops January 1 2020, but the final release will occur after that date." means something like "We will accept new bug reports and may start fixing things until January 1; however, the fixes in progress as of that date will be completed and released in a final version some time later." When we EOL py27 support for Twisted we should probably do something similar so anyone who wants to get a pending fix out as part of the last 27-supporting release has a window to do that. -g
![](https://secure.gravatar.com/avatar/c312623edb9666f92c17bf2c925947f4.jpg?s=120&d=mm&r=g)
On Dec 16, 2019, at 7:10 AM, Glyph <glyph@twistedmatrix.com> wrote:
When we EOL py27 support for Twisted we should probably do something similar so anyone who wants to get a pending fix out as part of the last 27-supporting release has a window to do that.
That seems pretty reasonable (as long as it's not open-ended). -wsv
![](https://secure.gravatar.com/avatar/3d37232726396a1d3c7412dd915095ea.jpg?s=120&d=mm&r=g)
On 17/12/19 3:40 am, Glyph wrote:
At this point, I think 1st of January is unrealistic for those that want to do a final drive for things inclusive of 2.7 support. Without any better ideas, what if we just say something like 1st March? I don't think much would realistically happen in the rest of March, considering a fair amount of people would be gearing up for PyCon, and then it lets us have a decent amount of space for a release candidate and the potential for any successive RCs without it spilling over into PyCon US, where I have a feeling those of us attending would prefer to look to the future :) - Amber
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Given the slightly different way our release process works, my understanding of this proposal is "the first release cut after 2020-3-1 will be the last one to support Python 2.7; after that release is cut, non-2.7-compatible work will begin occurring on trunk". If my understanding is correct, I second it. 3 months after the official EOL seems like enough time for people who really care about legacy support to get their fixes in. Forward! -g
![](https://secure.gravatar.com/avatar/8285ea1765ae52a3ab79c767676a2f4e.jpg?s=120&d=mm&r=g)
On 12/11/2019 4:06 PM, Wilfredo Sánchez Vega wrote:
WHAT????? You are saying that I will still have to use a version of Twisted that supports Python 2.7, even though I AM NOT USING Python 2.7? If that's not what you meant, than clarify.
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
On Wed, Dec 11, 2019, at 1:06 PM, Wilfredo Sánchez Vega wrote:
Note that multipart requests (relevant for Trac if we want to support ticket attachments) are broken on Python 3.7+ due to some questionable changes to the cgi module. We may consider this blocking if we aim to transition to 3.7+. There is a fix here: https://github.com/twisted/twisted/pull/1207
I have a fix for this ready for review here: https://github.com/twisted/twisted/pull/1212 ---Tom
![](https://secure.gravatar.com/avatar/607cfd4a5b41fe6c886c978128b9c03e.jpg?s=120&d=mm&r=g)
On Wed, Dec 11, 2019 at 4:06 PM Wilfredo Sánchez Vega <wsanchez@wsanchez.net> wrote:
With Python 2.7 support dropping in a matter of weeks, I'd like to pick up from hawkowl's thread back in March about Python 2.7 support for Twisted.
Despite the widely circulated January 1 2020 date, it seems that python-dev nevertheless plans to make a Python 2.7 release in April 2020 - https://www.python.org/dev/peps/pep-0373/ This is not to say the questions here don't need answering or that there isn't some urgency at this point, but it seems like the January 1 date is essentially an arbitrary deadline without a clear connection to the support status of CPython 2.7. Or does anyone know a different way to reconcile these apparently conflicting dates? Jean-Paul
![](https://secure.gravatar.com/avatar/c312623edb9666f92c17bf2c925947f4.jpg?s=120&d=mm&r=g)
I don't object to the final release being in April instead of January, though I'm not sure that the additional release really means we need to do that. But FWIW I'm in favor of Option 3 / calling 19.4 the final release. :-) My sense is that there was consensus on hawkowl's proposal except for, I think: 1- Should we drop the idea of a 2.7 maintenance branch? 2- It seems we need to fix the bug in twisted.names first. 3- Choosing a final 2.7 release date. -wsv
![](https://secure.gravatar.com/avatar/3d37232726396a1d3c7412dd915095ea.jpg?s=120&d=mm&r=g)
On 12/12/19 8:41 am, Wilfredo Sánchez Vega wrote:
JP has mentioned elsewhere that enough of the ecosystem has dropped, or will drop, support that a lot of applications staying on 2.7 will have dependencies frozen in time regardless. I think this is valid, and that Twisted remaining compatible with 2.7 hardly matters when so much else isn't and forces the hand of 2.7-stalwarts regardless. As such, I think dropping the 2.7 maint branch is valid. And yes, we do need to fix that bug. There's some other py3 specific bugs in names's host file resolver, but, I think that bugfix would end up hitting that portion of code's unicode uncleanliness. It just comes down to someone doing it, I guess... - Amber
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
This is just speculative, just based on my own historical understanding of python-dev's process and somewhat informal definition of "support", but I believe the sentence "Support officially stops January 1 2020, but the final release will occur after that date." means something like "We will accept new bug reports and may start fixing things until January 1; however, the fixes in progress as of that date will be completed and released in a final version some time later." When we EOL py27 support for Twisted we should probably do something similar so anyone who wants to get a pending fix out as part of the last 27-supporting release has a window to do that. -g
![](https://secure.gravatar.com/avatar/c312623edb9666f92c17bf2c925947f4.jpg?s=120&d=mm&r=g)
On Dec 16, 2019, at 7:10 AM, Glyph <glyph@twistedmatrix.com> wrote:
When we EOL py27 support for Twisted we should probably do something similar so anyone who wants to get a pending fix out as part of the last 27-supporting release has a window to do that.
That seems pretty reasonable (as long as it's not open-ended). -wsv
![](https://secure.gravatar.com/avatar/3d37232726396a1d3c7412dd915095ea.jpg?s=120&d=mm&r=g)
On 17/12/19 3:40 am, Glyph wrote:
At this point, I think 1st of January is unrealistic for those that want to do a final drive for things inclusive of 2.7 support. Without any better ideas, what if we just say something like 1st March? I don't think much would realistically happen in the rest of March, considering a fair amount of people would be gearing up for PyCon, and then it lets us have a decent amount of space for a release candidate and the potential for any successive RCs without it spilling over into PyCon US, where I have a feeling those of us attending would prefer to look to the future :) - Amber
![](https://secure.gravatar.com/avatar/e1554622707bedd9202884900430b838.jpg?s=120&d=mm&r=g)
Given the slightly different way our release process works, my understanding of this proposal is "the first release cut after 2020-3-1 will be the last one to support Python 2.7; after that release is cut, non-2.7-compatible work will begin occurring on trunk". If my understanding is correct, I second it. 3 months after the official EOL seems like enough time for people who really care about legacy support to get their fixes in. Forward! -g
![](https://secure.gravatar.com/avatar/8285ea1765ae52a3ab79c767676a2f4e.jpg?s=120&d=mm&r=g)
On 12/11/2019 4:06 PM, Wilfredo Sánchez Vega wrote:
WHAT????? You are saying that I will still have to use a version of Twisted that supports Python 2.7, even though I AM NOT USING Python 2.7? If that's not what you meant, than clarify.
-- John Santos Evans Griffiths & Hart, Inc. 781-861-0670 ext 539
![](https://secure.gravatar.com/avatar/bcb6ef473ff1644fddee1b4e7c730b01.jpg?s=120&d=mm&r=g)
On Wed, Dec 11, 2019, at 1:06 PM, Wilfredo Sánchez Vega wrote:
Note that multipart requests (relevant for Trac if we want to support ticket attachments) are broken on Python 3.7+ due to some questionable changes to the cgi module. We may consider this blocking if we aim to transition to 3.7+. There is a fix here: https://github.com/twisted/twisted/pull/1207
I have a fix for this ready for review here: https://github.com/twisted/twisted/pull/1212 ---Tom
participants (7)
-
Amber Brown
-
Amber Brown (hawkowl)
-
Glyph
-
Jean-Paul Calderone
-
John Santos
-
Tom Most
-
Wilfredo Sánchez Vega