Mailman 3 December 2019 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince 03 Mar '22

03 Mar '22

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 27 Oct '21

27 Oct '21

5 5

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] The Python 2.7 Plan
by Wilfredo Sánchez Vega 31 Dec '19

31 Dec '19

With Python 2.7 support dropping in a matter of weeks, I'd like to pick up from hawkowl's thread back in March about Python 2.7 support for Twisted. hawkowl had shared a plan in a Google Docs document. That URL has since gone stale but here's a new one (thanks, hawkowl): https://docs.google.com/document/d/1lIkg6mYVMxVhh3_U6uSJ1XjllPM92Nf8Zp7b7OZ… Here's what I got from the prior conversation: Amber Brown (hawkowl): > I personally find that Python 2 compat adds a huge amount of overhead > when working on and maintaining Twisted, and think that with the current > maintainer availability, dropping it sooner rather than later would have > a beneficial effect on how much work we spend on shims/compat, > complexity, and our ability to ship new features, as well as onboarding > people who are interested in the project, but have no interest (or > experience!) in Python 2.7. As an example here, I'd like us to start adopting type hints. I find them incredibly valuable as documentation and for catching lots of bugs. The best way to adopt them is to use inline syntax supported by Python 3.6+, or slightly less ideal, Python 3.5+. It looks like pydoctor may have support for this soon, which makes that an increasingly interesting avenue. To support Python 2.7, we'd have to use either: - the comments-based inline syntax, which is far uglier and leaves us with an extra project of converting those later when we finally do drop 2.7, or - stub files, which are considerably less easy to manage over time Doing this will also afford us an opportunity to better express the API (do we mean to return a "mutable sequence" or a specifically a list, etc.). There's a longer conversation to be had about type hints, and that should be on a separate thread; I only talk about it here to explain why Python 2.7 is in the way. > One of my rationales is that from some analysis of PyPI download > statistics, the vast majority of Python 2 users are using old versions > of Twisted, while nearly all our Python 3 users are on the latest > version. Can you share how you looked up these stats? I don't see a way to do so with pypistats. John Santos: > I hope/expect that none of my code changes to make my source compatible > with both 2.7 and 3 will break when Twisted support for 2.7 goes away! > (It seems to me the Twisted compatibility policy would guarantee that.) To be clear, I would not expect this to be the case if you update Twisted to a version that no longer supports Python 2.7. You would have to continue using a version that does support Python 2.7. You should have some runway, however, in which your app still works with either Py2.7 and Twisted 19 as well as Py3.8 using Twisted 20, for example. Glyph, in reference to a maintenance branch for Python 2.7: > Personally, I'm not willing to commit to this. I know from experience both on Twisted and other projects that maintaining multiple release branches, even one that's "maintenance only", requires at least one point-person for each branch at all times (usually a "release manager"). And backporting fixes inevitably gets harder and harder as the "maintenance" branch diverges from top-of-tree. If I have time to work on Twisted in my increasingly scarce spare time, I want it to be on something at least plausibly interesting, and manually backporting manual fixes to an unmaintained py2 branch that I don't even use doesn't qualify. I tend to agree. I think the compromise would be that we could be willing to accept back-ports of certain fixes (presumably limited to security fixes?) to a maintenance branch, but not that we would commit to doing so for any period of time. Additionally, we've already supported Python 2.7 for the entirely supported lifetime of Python 2.7. Glyph also notes some infrastructure we use that is blocked on code fixes: - Trac: https://trac.edgewall.org/ticket/12130 Looks like we can leave the Trac instance running in a Twisted 19 WCGI server until they fix that. - DNS: https://twistedmatrix.com/trac/ticket/9496 I'd suggest we do the same as with Trac until this is fixed, but Glyph disagrees. Back to hawkowl's document: - Part 1: I think we should drop the Py2.7 maintenance branch piece per Glyph's comment above. - Part 2: Re-introduced modules should go into separate repositories, not back into Twisted proper. As to the timeline, Option 1 would be to cut a 2.7 release in a couple of weeks. I don't know that we need to do this in January, but I would suggest that our next release is the final release for 2.7. This would require a fix to the DNS issue #9496. -wsv

7 11

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 29 Dec '19

29 Dec '19

1 0

[Twisted-Python] Conch Ed25519: testing advice wanted
by Colin Watson 23 Dec '19

23 Dec '19

Hi, I've been working for a while on adding Ed25519 SSH key support to Conch. This is almost done: if and when https://github.com/twisted/twisted/pull/1210 lands, everything should work provided that you're using the cryptography wheel or otherwise have a new enough OpenSSL to support X25519 (for curve25519-sha256 key exchange) and Ed25519 (for the key format itself). My professional interest in having this feature is for Launchpad, and in that case we want to use the Ubuntu-packaged OpenSSL (since we're in the same company as the security team supporting it, after all). The awkward thing is then that we're currently on Ubuntu 16.04, which has OpenSSL 1.0.2g, which is too old to support either X25519 or Ed25519. Ubuntu 18.04 has fortunately been updated to OpenSSL 1.1.1, which supports X25519, but cryptography's Ed25519 support requires OpenSSL 1.1.1b. I've asked about getting 18.04 updated to 1.1.1b or newer, but as I understand it, despite the innocuous-looking change in version number, this is a pretty complicated endeavour and may well not happen. In https://github.com/twisted/twisted/pull/644, I brought up the possibility of using PyNaCl as an alternative implementation to cope with this sort of case, and Glyph seemed generally supportive of that idea, so I've been looking into the details. Using PyNaCl as an X25519 alternative seems unfortunately difficult, as it only exposes a wrapped version of X25519 which isn't quite what we need, so I'm assuming that I'll need to update to Ubuntu 18.04 and have been confining my investigations to just the Ed25519 compatibility portion. The actual code in this case isn't difficult: it's only 50 lines or so for versions of nacl.signing.SigningKey and nacl.signing.VerifyKey which implement cryptography's Ed25519PrivateKey and Ed25519PublicKey interfaces. The bit I'm stuck on is how best to test this. Of course I can easily enough make some tests pretend that cryptography doesn't support Ed25519, but Twisted doesn't currently depend on PyNaCl, and it doesn't need it for anything other than this edge-case compatibility layer. I can see a few options: * Make Twisted[conch] depend on PyNaCl, and use that to test the compatibility layer. This would work, but it seems a bit rude. * Add extra test matrix entries with different dependencies to test the PyNaCl-based compatibility code. This seems kind of excessive and awkward, not to mention fiddly to maintain just for this. * Add a Twisted[test] extra or similar for things that are only needed by the test suite. * Just don't test the compatibility layer. I don't imagine this would be a popular option. Any preferences, or any other suggestions? -- Colin Watson [cjwatson(a)debian.org]

1 1

[Twisted-Python] operational visibility
by Glyph 22 Dec '19

22 Dec '19

At twistedmatrix.com <http://twistedmatrix.com/>, we dogfood a bunch of Twisted software: our DNS server (primary and secondary!), our web server, even a little bit of email stuff here and there. And we have our own structured logging system to make it, ahem, "easy" to put logs into some kind of a system that can tell you what Twisted is doing. But uh… we don't get a lot of value from this because the tracebacks just go into disk files that nobody ever looks at. Does anyone know if, say, Sentry supports a free tier for open source projects, or something like that? Or any other error-tracking provider even vaguely compatible with Twisted might do something like that? It would be nice to surface and fix tracebacks that affect our own infrastructure before they hit our users. -g

2 3

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 22 Dec '19

22 Dec '19

1 0

[Twisted-Python] Weird epoller bug
by Robert DiFalco 17 Dec '19

17 Dec '19

We recently switched from these versions and recently our TLS health check using treq seems to be using more and more file handles and getting this exception. Additionally our health check gets response time outs at a much higher frequency. Any tips on how I would go about debugging this? Or is this perhaps a known issue in one of the versions below and I need to either bump up or down a release version? Thanks! twisted 16.6.0 -> 19.19.0 treq 15.1.0 -> 18.6.0 File "/home/foo/foo/local/lib/python2.7/site-packages/twisted/internet/epollreactor.py", line 183, in removeWriter EPOLLOUT, EPOLLIN) File "/home/foo/foo/local/lib/python2.7/site-packages/twisted/internet/epollreactor.py", line 160, in _remove self._poller.unregister(fd) exceptions.IOError: [Errno 2] No such file or directory

1 2

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 15 Dec '19

15 Dec '19

1 0