Mailman 3 - Twisted - {{ block.super }

review needed: better type hints for FilePath, Deferred
by Glyph 21 Mar '23

21 Mar '23

I've got two PRs up right now which pretty significantly enhance Twisted's type annotations in two core areas: FilePath and Deferred. In FilePath's case, things were previously entirely un-annotated. With the PR I have up, FilePath is now generic, i.e. FilePath[str] or FilePath[bytes] so you can clearly see what the type of `.path` (and friends) are and get proper type checking. I'm writing some FilePath intensive code in an application right now and this is really handy and has caught several minor bugs for me already. (And because I've gotta make CI happy, this includes ZipPath…) https://github.com/twisted/twisted/pull/11823 #11822 filepath annotations by glyph · Pull Request #11823 · twisted/twisted github.com For Deferreds, there are just a bunch of places where type-checking silently fails; for example, addCallback and addErrback don't check their arguments at all. Perhaps more saliently to modern code, Deferred.fromFuture() wasn't specializing its result, so you'd lose all type-checking across an asyncio boundary. And maybeDeferred() just returned a nonsense result of Deferred[Deferred] so it was effectively unusable. https://github.com/twisted/twisted/pull/11770 #11772 Correct deferred annotations by glyph · Pull Request #11770 · twisted/twisted github.com Both are passing tests, have full coverage, and have had at least a little bit of feedback, so they should be relatively easy reviews. If you've got some application code that you would like to type-check against these to see if they spot any bugs for you, that would be great, but if we I could get these approved, this type of comprehensive type-safety upgrade would make a nice tentpole for our first 2023 release. -g

2 2

posix_spawn would really speed up spawnProcess if we used it
by Glyph 16 Mar '23

16 Mar '23

I am writing some code that was spawning a lot of processes with spawnProcess, and it was going a bit slower than I expected; it was taking about 51 seconds on my M1 Max mac. So I hopped over to my local copy of Twisted and applied this patch: 1 file changed, 9 insertions(+) src/twisted/internet/process.py | 9 +++++++++ modified src/twisted/internet/process.py @@ -367,6 +367,15 @@ class _BaseProcess(BaseProcess): @type environment: L{dict}. @param kwargs: keyword arguments to L{_setupChild} method. """ + from os import posix_spawn, POSIX_SPAWN_DUP2, POSIX_SPAWN_CLOSE + fdmap = kwargs.get("fdmap", {}) + actions = [ + (POSIX_SPAWN_DUP2, childFD, parentFD) + for (parentFD, childFD) in fdmap.items() + ] + self.pid = posix_spawn(executable, args, environment, file_actions=actions) + return + collectorEnabled = gc.isenabled() gc.disable() try: This is obviously not a complete reimplementation and might have some slight performance advantage due to its incompleteness, but the task in question then took 16 seconds; i.e. a 319% speedup. Anyone want to do this implementation for real? Or want to commit to a review, if I did it? -g

2 3

Weird "bugs" in EPollReactor? You may be using the reactor from a thread by mistake
by Chris Withers 15 Mar '23

15 Mar '23

Hi All, So, this was "fun"... Eventually tracked down a bug in Twisted based application that uses Autobahn for websockets, some AMP stuff and a good amount of other Twisted stuff, but that has to call a bunch of blocking code, some of it written by users of the app, so we want to insulate them from thinking about Deferreds or anything async. We saw weird KeyErrors, ENOENT, "Reactor was unclean" during shutdown in trial-based test suites and generally head-in-sanded it or grumbled about CI instability due to the environment in which the tests were running. This went on for a couple of years... no kidding... Just recently, finally spotted the problem... a bunch of handler code was being deferToThread'ed so that it didn't accidentally block the reactor, BUT it was also calling sendMessage [1]... Big "d'oh" when that penny finally dropped... Moving that code to using blockingCallFromThread magically fixed all the "reactor bugs" and "CI environment instability", so wanted to write something up here for future self or google hunters... More over on Stack Overflow: https://stackoverflow.com/a/75674510/216229 cheers, Chris [1] https://autobahn.readthedocs.io/en/latest/reference/autobahn.websocket.html…

1 0

treq digest_auth
by jjaherne＠gmail.com 13 Mar '23

13 Mar '23

I'm not sure tis is the right p[lace to ask this question but here goes. I see that digest_auth was added to treq last November/December. I can't see any way to get a copy of the code so I can run some tests to see if it fixes a problem I have. Anyone point me in the right direction.

2 2

Twisted Azure backups and VM removed
by Adi Roiban 18 Feb '23

18 Feb '23

Hi, Just FYI. Today I found out that on the 26 of September 2022 the Twisted Visual Studio Azure subscription that we had as a sponsorship from MS was disabled. On the 30 of December 2022 the whole "resources" were removed. This means that we lost the backups from the old Dornkirk VM. Also, all the services migrated from Dornkirk are no longer available. Most services were already migrated from Azure VM, so in terms of services, the important disruption is in terms of the DNS server. The backups could also be an important part, but I think that the important information was archived in various Twisted GitHub repos. I don't think that it is worth trying to recover the data. I was expecting to receive a separate email from MS to inform me that they will no longer support Twisted, and not just a generic Azure subscription disable message. The Twisted Azure subscription message was lost between my other messages, as in the same period I was also disabling/removing other Azure subscriptions :( Regards -- Adi Roiban

2 2

PyDoctor as generator on ReadTheDocs?
by c.buhtz＠posteo.jp 30 Dec '22

30 Dec '22

Hi, I'm using pydoctor locally. ReadTheDocs.org do use Sphinx by default. But looking into the docs it seems to me that it should be possible to use every other tool to generate HTML out of your source code. Am I right? Does anyone knows a project using PyDoctor on ReadTheDocs? This question is specific to PyDoctor without(!) Sphinx involved. I do know that even youself do use Sphinx for the PyDoctor docu instead of using pure-PyDoctor. No Sphinx! Sphinx would not be an option not even in combination with something else. ;) Maybe asking the other way around: Are there alternatives to ReadTheDocs? Kind Christian

3 5

Re: Help to unsubscribe
by Glyph 31 Oct '22

31 Oct '22

Done. Not sure what the issue was, people successfully subscribe and unsubscribe all the time, but it might be worth noting that mail.python.org(http://mail.python.org) hosts both a mailman2 and a mailman3 instance, and the twisted list is only on the mailman3 one. On October 31, 2022 at 10:19:13 AM, Mikhail Terekhov (termim@gmail.com(mailto:termim@gmail.com)) wrote: > Hello, > > I'm trying to unsubscribe from this list but keep getting messages from > the mailman that my email is not known. > > Could someone with admin privileges help me unsubscribe. My email is > termim(a)gmail.com > > > Thanks, > > Mikhail > > _______________________________________________ > Twisted mailing list -- twisted(a)python.org > To unsubscribe send an email to twisted-leave(a)python.org > https://mail.python.org/mailman3/lists/twisted.python.org/ > Message archived at https://mail.python.org/archives/list/twisted@python.org/message/T2OS4O5IS2… > Code of Conduct: https://twisted.org/conduct

1 0

Help to unsubscribe
by Mikhail Terekhov 31 Oct '22

31 Oct '22

Hello, I'm trying to unsubscribe from this list but keep getting messages from the mailman that my email is not known. Could someone with admin privileges help me unsubscribe. My email is termim(a)gmail.com Thanks, Mikhail

1 0

Twisted 22.10.0 Final Release Announcement
by Adi Roiban 31 Oct '22

31 Oct '22

Hi all, Twisted 22.10.0 was released It is available for download from PyPi https://pypi.org/project/Twisted/22.10.0/ Release notes on GitHub https://github.com/twisted/twisted/releases/tag/twisted-22.10.0 ---------------- There are no changes since the first release candidate. --------------- Many thanks to everyone who had a part in Twisted development, the supporters of the Twisted Software Foundation, the developers, and all the people testing and building great things with Twisted! Slava Ukraini! -- Adi Roiban

2 1

Twisted 22.10.0 Pre-Release Announcement
by Adi Roiban 28 Oct '22

28 Oct '22

On behalf of the Twisted contributors, I announce the release candidate of Twisted 22.10.0 This release was triggered by CVE-2022-39348 https://github.com/twisted/twisted/security/advisories/GHSA-vg46-2rrj-3647 The release and NEWS file is available for review at https://github.com/twisted/twisted/pull/11732/files Release candidate documentation is available at https://twisted--11732.org.readthedocs.build/en/11732/ Wheels for the release candidate are available on PyPI https://pypi.org/project/Twisted/22.10.0rc1/ python -m pip install Twisted==22.10.0rc1 Please test it and report any issues. If nothing comes up in one week, I will do the final release based on the latest release candidate. Many thanks to everyone who had a part in Twisted development, the supporters of the Twisted Software Foundation, the developers, and all the people testing and building great things with Twisted! Slava Ukraini! -- Adi Roiban

3 8