Mailman 3 - Twisted - python.org

native websockets support available for review
by glyph＠twistedmatrix.com April 18, 2025

April 18, 2025

Hello friends, The native websockets branch that I've been puttering with is now available for review, so I'm asking for someone to review it: https://github.com/twisted/twisted/pull/12397/files I'm excited to get this landed and released. WebSockets are probably the most popular and common form of bidirectional asynchronous communication happening on the internet today, and it's been a bit of a weakness that we don't have a simple showcase for that in Twisted. IMAP and SSH are great to have, but they're not the first thing people go looking for these days when they're looking to do something async. We have of course had some access to the WebSockets world via Autobahn, but first-party integration will have some significant advantages: We can put a WebSockets example onto our minimal-examples showcase on twisted.org, because the example is similarly very small, and now included with no imports from other libraries. This is the major thing; a dozen or so lines of code illustrating how to push data from Twisted into a browser that a user can immediately interact with is an important missing piece of the pitch of Twisted both as a library and as a web server. There will be a much simpler onboarding experience to this functionality. Autobahn is still there for anyone who wants WAMP or XBR, but users won't be presented with those confusing concepts when just trying to do simple server-side push. Since the gross private API internals that we rely upon are our own, we don't have the version-skew problems that Autobahn does <https://github.com/crossbario/autobahn-python/blob/7bc85b34e200640ab98a41cf…>, nor do we need overhead like anyio. Since this is a much more recent implementation it also comes with modern niceties like coroutines rather than inlineCallbacks, comprehensive type annotations, and 100% code coverage. This should also be a pretty easy layup of a review: There is full code coverage and I believe full public API documentation coverage. There's new narrative documentation <https://twisted--12397.org.readthedocs.build/en/12397/web/howto/web-in-60/w…> explaining how to use it. The protocol implementation itself is delegated to the existing, minimal, sans-io wsproto library <https://github.com/python-hyper/wsproto> so there's not a lot of protocol implementation stuff to review. The tests are all the new, good kind (i.e.: no native sockets, no real reactors), using IOPump (and improving that infrastructure a bit as I passed through it), with (hopefully!) no new flakiness or weird slowdowns. The downside is that it is brushing up against 1500 changed lines, with most of that being additions, so it is on the longer side. Thanks in advance for volunteering, -g

3 4

ANN: Pyng monitoring released
by Kirill Miazine Feb. 4, 2025

Feb. 4, 2025

Hi, all The last couple of weeks I had been writing docs for my monitoring and alerting system written using Twisted called Pyng. I consider Pyng finished software -- Twisted gives me confidence that it will continue to run in 20 years. Please see https://uptime.is/pyng/ for details. Regards -- Kirill

2 1

reactor.running False in (some) callback/errback
by Kirill Miazine Jan. 28, 2025

Jan. 28, 2025

This has been driving me crazy for a while -- for some reason reactor.stop() in the _error errback in example below raises error.ReactorNotRunning. In order to stop the reactor, I have to do reactor.callWhenRunning(reactor.stop) (or I did reactor.callLater(0, ...) until I discovered callWhenRunning). In the example, I bind to a low port to make sure error is triggered. #!/usr/local/bin/python3 from twisted.internet import reactor, defer, protocol from twisted.internet.protocol import Factory from twisted.internet.endpoints import TCP4ServerEndpoint def _port(port): print('got', port) def _error(err): print('got err', err) print('is reactor running?', reactor.running) print('is reactor running?', (lambda: reactor.running)()) reactor.stop() # reactor.callWhenRunning(reactor.stop) d = TCP4ServerEndpoint(reactor, 123).listen(Factory()) d.addCallback(_port) d.addErrback(_error) d.addErrback(print) reactor.run() OTOH, in the following example reactor.stop() is stoppig the reactor properly: #!/usr/local/bin/python3 from twisted.internet import reactor, defer def cb(res): print('running?', reactor.running) if res == 'bar': raise Exception() reactor.stop() def eb(err): print('running?', reactor.running) print(err) reactor.stop() d = defer.Deferred() d.addCallback(cb) d.addErrback(eb) #reactor.callWhenRunning(d.callback, 'foo') reactor.callWhenRunning(d.callback, 'bar') reactor.run() Any ideas?

3 5

(tiny) twisted success story
by Kirill Miazine Jan. 22, 2025

Jan. 22, 2025

Hello, world! I'm very happy to be using Twisted again -- after a break of about 20 years! In ol' days I did mailing lists in Twisted, implemented Dovecot AUTH protocol in Twisted and a lot more. Mostly it was mail related. A lot have been forgotten and have to be re-discovered. A couple of words about use-case: I was curious about what could be achieved using the filtering facility in the OpenSMTPD MTA. The filters are programs which are executed by the daemon, and some data is exchanged on stdin/stdout using non-blocking fashion. More at https://man.openbsd.org/smtpd-filters I tried getting Python asyncio to work at first, even using aioconsole package, but I kept hitting something which made the thing break after handshake exchange. This evening I did a rewrite in Twisted, it took me a couple of hours, but the was running on the first try! Thanks for all the fish! -- Kirill

3 6

time-sensitive code reviews
by Glyph Jan. 20, 2025

Jan. 20, 2025

As per this announcement <https://about.readthedocs.com/blog/2024/12/deprecate-config-files-without-s…>, several of our projects' readthedocs builds will stop working tomorrow. I have prepared 3 PRs to address this: https://github.com/twisted/twisted/pull/12420 https://github.com/twisted/treq/pull/406 https://github.com/twisted/towncrier/pull/681 The Twisted one also includes a version-specification fix for PyObjC on 3.8, because that will also make trunk fail right now if tests were to run on it. (We should really pin our deps, but I digress.) These should be very easy code reviews for anyone interested, and it would be great to get these done before all the builds start failing tomorrow. Thanks, -g

1 1

PB: remote_* deferred return magic
by Kirill Miazine Jan. 11, 2025

Jan. 11, 2025

I'm playing with PB, and it's booth cool and somewhat confusing. So I have some remote_* methods, e.g. def remote_ping(self): return 'pong' def remote_run(self): d = Deferred() d.AddCallback(self.got_result) # return d # <- magic here when deferred is returned? def got_result(self, res) return res When I do callRemote('ping') and execute callback, I am getting 'pong' as a result. However, when I do callRemote('run'), it returns a deferred which returns None, which is expected, as remote_run() doesn't have any result yet. However, if I return deferred ("d") from remote_run, then callRemote('run') gives me the result. Is it some expected magic going on when remote_ method returns a deferred?

4 5

typo on "Configuring and Using the Twisted Web Server"
by Kirill Miazine Dec. 29, 2024

Dec. 29, 2024

While browsing through the twisted.web docs, I noticed following: Hello's getChild() method on https://docs.twisted.org/en/stable/web/howto/using-twistedweb.html#resource… has a typo. It says if name == '': where it should have said: if name == b'': Greetz Kirill

3 4

INCOMPATIBLE CHANGE: Removal of TestCase.defer{SetUp, TestMethod, TearDown, RunCleanups} methods
by povilas＠radix.lt Dec. 13, 2024

Dec. 13, 2024

Hi, I created this PR to remove TestCase.defer{SetUp, TestMethod, TearDown, RunCleanups} methods: https://github.com/twisted/twisted/pull/12389 The justification is as follows: Currently `TestCase` has the following functions that are presumably public due to their naming: `deferSetUp`, `deferTestMethod`, `deferRunCleanups` and `deferTearDown`. This raises a few problems. I suggest removing the functions without deprecation period due to the following reasons: - these functions expose so much of the internal structure of TestCase that it makes the code hard to change (e.g. https://github.com/twisted/twisted/pull/12375 cannot go in) - there is no case where the functions would be useful as an extension point. The only use of the functions that I found using Github code search is my own code in the Buildbot project which overrides `deferRunCleanups` and hooks even deeper into TestCase internals. - the API is not "public ready" (i.e. the `ignored` parameter) - it seems as if they were some kind of internal thing that simply didn't get the `_` prefix. - deprecation would be harder to implement than usual, due to the functions being supposed to be overridden, not just called. - this is API for tests, so it is not likely to break functionality that is used in production. Only development workflows may be affected where the fix is to update the tests. Given the above - small risk of breaking code for someone and relatively high costs for going through the deprecation dance I would suggest to have an exception for this change and allow to remove the functions without deprecation period. If you have any objections and you think that the functions should stay or at least be removed via deprecation please let me know. You can send your feedback by replying to this email or add your comment to the PR https://github.com/twisted/twisted/pull/12389 Thanks for your time! Regards, Povilas

4 6

Twisted 24.11.0 release candidate
by Adi Roiban Nov. 23, 2024

Nov. 23, 2024

On behalf of the Twisted contributors I announce the release candidate of Twisted 24.11.0. This is a release triggered by the following bugfixes: - twisted.internet.endpoints.clientFromString can handle "tls:" endpoints with default `bindAddress`. This was a regression introduced in 24.10.0. (#12350) - HTTP server can handle listening on filesystem UNIX sockets. This was a regression introduced in 24.10.0. (#12365) - twisted.internet.tcp.Connection.setTcpNoDelay was updated to fix a potential failure when setting TCP_NODELAY on macOS. (#12372) The release and NEWS file is available for review at https://github.com/twisted/twisted/pull/12374 Release candidate documentation is available at https://twisted--12374.org.readthedocs.build/en/12374/ Wheels for the release candidate are available on PyPI python -m pip install Twisted==24.11.0rc1 Please test it and report any issues. If nothing comes up in the next few days, 24.11.0 will be released based on the latest release candidate. I would like to use this opportunity and thank https://thinkst.com/ and https://www.sftpplus.com/ for the ongoing financial support. -- Adi Roiban

2 1

Twisted Vulnerabilties
by csoc＠inl.gov Nov. 4, 2024

Nov. 4, 2024

I'm currently looking into a couple of twisted vulnerabilities and came across the following vulnerabilities: https://security.snyk.io/vuln/SNYK-PYTHON-TWISTED-3040866 https://security.snyk.io/vuln/SNYK-PYTHON-TWISTED-564357 Does anyone know why these weren't given CVEs?

2 1