Mailman 3 September 2020 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince 03 Mar '22

03 Mar '22

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 27 Oct '21

27 Oct '21

5 5

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] code contribution/participation policy
by Boris Epstein 10 Mar '21

10 Mar '21

Hi all, I have started working with Twisted and found it to be a very powerful package. That said, some modules are not documented very thoroughly (big surprise in the world of software, right?) and I also found some questionable code, too. If I were to contribute fixes/improvements, what is the submission procedure? Thanks. Boris.

4 8

[Twisted-Python] smb component progress
by Ian Haywood 25 Nov '20

25 Nov '20

I have continued to work slowly on this project. Named pipes are supported, the samba client can connect and list available shares. i am currently working on a vfs layer using deferToThread to provide asynchronous file access I have tried to keep the code divided into logically separate chunks . my first chunk received comments from glyph 2 months ago but has remained in the review queue since. https://twistedmatrix.com/trac/ticket/9818 is there anyone available to review this code to progress it ? Ian

2 1

[Twisted-Python] Request for new Twisted release?
by Craig Rodrigues 12 Oct '20

12 Oct '20

Amber, Can we have a new Twisted release within the next two months, say in Nov./Dec. timeframe, or sooner if you'd like? In Twisted trunk, there are a lot of things that would be good to have in a new release. Here are some of the high order items: *Python 3 Fixes* - twist dns --pyzone now works on Python 3 (I had a project 6 months ago where I would have liked to have this in a release) - twisted.web.twcgi now works on Python 3 *Python 3.8 Fixes* - all tests pass on all platforms on Python 3.8, including asyncio tests on Windows *Python 3.9 Fixes* - all tests pass on Python 3.9.0rc2 (at least on Mac and Linux), (thanks to Michał Górny) *Python 2.7* - Python 2.7 support has been dropped in trunk *Black* - black code formatter has been integrated into the CI for Twisted, and all Twisted code is now formatted with Black (thanks to Tom Most) *Mypy* - mypy is now run as part of Twisted CI and checks the code for every PR. Type annotations compatible with Python 3.5+ are slowly being added to the codebase. There are a lot of other small fixes and enhancements that people have contributed since the last Twisted release. I would like this new release to be the last release supporting Python 3.5, so that Python 3.5 support can be removed in trunk, and PEP-526 style variable annotations can be added in trunk. Thanks for your consideration. -- Craig

5 12

[Twisted-Python] Twisted with pypy3 on Ubuntu 20.4, SSL error: 'ee key too small'
by Craig Rodrigues 30 Sep '20

30 Sep '20

Hi, I ran an experiment to run the Twisted tests on Ubuntu 20.4 using this version of using pypy3: *Python 3.6.9 (2ad108f17bdb, Apr 07 2020, 02:29:05* *Pypy 7.3.1 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)* A bunch of the conch tests failed in this Ubuntu environment: https://github.com/twisted/twisted/runs/1173397508 like this: *===============================================================================Error: Traceback (most recent call last): File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/test/test_sslverify.py", line 1780, in test_ellipticCurveDiffieHellman onData=onData, File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/test/test_sslverify.py", line 716, in loopback self.serverPort = reactor.listenSSL(0, serverFactory, serverCertOpts) File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/internet/posixbase.py", line 593, in listenSSL tlsFactory = tls.TLSMemoryBIOFactory(contextFactory, False, factory) File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/protocols/tls.py", line 748, in __init__ contextFactory = _ContextFactoryToConnectionFactory(contextFactory) File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/protocols/tls.py", line 629, in __init__ oldStyleContextFactory.getContext() File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/internet/_sslverify.py", line 1636, in getContext self._context = self._makeContext() File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/twisted/internet/_sslverify.py", line 1645, in _makeContext ctx.use_certificate(self.certificate) File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/OpenSSL/SSL.py", line 960, in use_certificate _raise_current_error() File "/opt/hostedtoolcache/PyPy/3.6.9/x64/lib_pypy/_functools.py", line 80, in __call__ return self._func(*(self._args + fargs), **fkeywords) File "/home/runner/work/twisted/twisted/build/alldeps-withcov-posix/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue raise exception_type(errors)OpenSSL.SSL.Error: [('SSL routines', 'SSL_CTX_use_certificate', 'ee key too small')]twisted.test.test_sslverify.OpenSSLOptionsECDHIntegrationTests.test_ellipticCurveDiffieHellman===============================================================================* I'm not sure what the root problem is, but my suspicion is that the SSL library in this particular Ubuntu environment does not permit keys smaller than 2048 bits. Anyone have any other ideas as to the cause of this error? Thanks. -- Craig

2 2

[Twisted-Python] doWrite on twisted.internet.tcp.Port
by chris＠cmsconstruct.com 30 Sep '20

30 Sep '20

Hey guys, Last year I hit a condition discussed in this ticket: https://twistedmatrix.com/trac/ticket/4759 for doWrite called on a twisted.internet.tcp.Port. I ignored it at the time since it was just on Linux, and my main platform was Windows. Now I'm coming back to it. I'll add context on the problem below, but first I want to ask a high-level, design-type question with multiprocessing and Twisted: Referencing Jean-Paul's comment at the end of ticket 4759, I read you shouldn't fork a process (multiprocessing module) that already has a Twisted reactor. Understood. But what about a parent process (not doing anything Twisted) forking child processes, where each child process starts their own Twisted reactor? Is that intended to work from the Twisted perspective? Context: I only hit this problem on Linux, not Windows. The software project (github.com/opencontentplatform/ocp) has been a lot of fun, especially with walking the tight rope in using multi-processing, multi-threading, and Twisted reactors. The main controller process kicks off about 10 child processes, each doing different types of work. In general though, the child processes individually start a Twisted reactor, connect to Kafka, connect to a database, use a shared REST API, and some listen for connecting clients to accomplish work. I test on Linux about once a year, so too many changes to rollback and figure out that way. It was working on Linux 2 years ago, but last year's testing and current testing, receive the doWrite error. It continues running fine on Windows. I've gone back about 2 years of versions with Python3, Twisted, and dependent libs. on both Windows and Linux. Every version change yields the same result - continues to work on Windows and continues to hit the error on Linux. So something I added has caused Linux to throw that error. I'm not explicitly sharing much between the main process and the sub processes. They are spun up with (1) a shared multiprocessing.Event() - to have the main process shut the children down, (2) their own unique multiprocessing.Event() - to have the child processes notify back to the parent, and (3) a deep copy of a dictionary (containing a bunch of settings that remain constant). The main process uses twisted.logger, but for testing I strip that out to remove any twisted imports in the main process. So I'm not importing anything Twisted in the main process, and I don't believe I'm explicitly sharing something I shouldn't. Seems like something is implicitly being exposed/shared across Linux child processes, that aren't on Windows. The tracebacks come through on Linux (sometimes randomly), on the console of the parent controller process. No need to paste here, since it's the same as the ticket shows. I can't reliably reproduce the problem, but I know if I stop/start client connections (ServerFactory/twisted.internet.interfaces.IReactorTCP and twisted.internet.protocol.ReconnectingClientFactory) then it will eventually happen. I need to devote time at whittling down the code and attempting to create a reliable test case. if even possible. The error is slightly different when running HTTP vs HTTPS, but the story is the same. It cripples whatever child process that hits it, from doing much of anything thereafter. Not much luck with troubleshooting. The tracebacks do not include a calling function from my code, to tell me where to start. And it happens across different child process types, so not the same one each time. When I throw debuggers on the child processes, the problem seems to mask itself. Well, at least I didn't hit the problem over the last 3 days using pudb and stepping through code at breakpoints. I'm absolutely open to suggestions for troubleshooting, but first wanted to take a HUGE step back and ask a design question regarding Twisted and multiprocessing. Thanks!

4 6

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 26 Sep '20

26 Sep '20

1 0

[Twisted-Python] Retry: Conch Issues
by Robert DiFalco 25 Sep '20

25 Sep '20

I am having troubles with Conch. I've been able to reproduce them in a unit test and all the code is here along with requirements.txt: https://github.com/radifalco/conch_issues I'm trying to reuse my connection, but if it is idle too long the server shuts it down. What I don't understand is why the next call to `FileTransferClient` does not call back or errback? It will just hang forever even though the Factory and Transport know the connection was closed (as evidenced by the log messages). I don't ever want to hang, if I use FileTransferClient I would like it always call a callback or errback. I feel like Conch is missing a deferred somewhere but damned if I can find it. Thanks!

3 5