Mailman 3 June 2019 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince March 3, 2022

March 3, 2022

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com Oct. 27, 2021

Oct. 27, 2021

5 5

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz Aug. 5, 2021

Aug. 5, 2021

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Twisted tips for designing highly concurrent twisted REST API
by Waqar Khan July 14, 2019

July 14, 2019

Hello folks, I recently stumbled upon twisted and was wondering if it could suit my needs. On one hand, I want to use python but on another hand there are all these scalability concerns with this language so, I though I would pick the brains of the community. So.. a flask based app would look something like this. similar_types = ['foo', 'bar', 'baz'] def long_computation(rec_type): # some long computation return recs @app.route('/fetch_similar_users/<user_id>' def fetch_similar_users(user_id) r = json.loads(requests.get('url/to/fetch/%s'%user_id).text) recs = {} for stype in similar_types: recs[stype] = long_computation(rec_type) return recs Now, I tried to "twistify" but it failed. @defer.inlinecallbacks def long_computation(rec_type): # some long computation *defer.returnValue(recs)* @defer.inlinecallbacks def fetch_data(user_id): r = yieldjson.loads(requests.get('url/to/fetch/%s'%user_id).text) defer.returnValue(r) @defer.inlinecallbacks def fetch_recs(user_id): data = yield fetch_data(user_id) recs = {} for stype in similar_types: d = defer.ToThread(fetch_data, *(stype)) rec = yield d recs[stype] = rec defer.returnValue(recs) I wrapped all the above in twisted render_Get method.. but then I did a load test with locust (https://docs.locust.io/en/latest/what-is-locust.html) framework. It choked. As the time progressed, the response time increased. I am guessing, things are still blocking. Can you please help me look into the right place. Why exactly am I seeing increase in response time as the time progresses. I am guessing things are still working in "blocking" fashion but i thought the above should run things in async. Thanks

5 7

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com June 30, 2019

June 30, 2019

1 0

[Twisted-Python] TB calling _IdentityTransferDecoder after decode finished.
by Scott, Barry June 27, 2019

June 27, 2019

I'm in the middle of trying to figure what causes this TB. Has this been seen before? File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite why = selectable.doRead() File "/usr/local/lib/python2.7/site-packages/twisted/internet/unix.py", line 191, in doRead return self._dataReceived(data) File "/usr/local/lib/python2.7/site-packages/twisted/internet/tcp.py", line 249, in _dataReceived rval = self.protocol.dataReceived(data) File "/opt/webdefence/lib/filtering_proxy_listener.py", line 144, in dataReceived return http.HTTPChannel.dataReceived(self, data) File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 2211, in dataReceived return basic.LineReceiver.dataReceived(self, data) File "/usr/local/lib/python2.7/site-packages/twisted/protocols/basic.py", line 579, in dataReceived why = self.rawDataReceived(data) File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 2218, in rawDataReceived self._transferDecoder.dataReceived(data) File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 1681, in dataReceived "_IdentityTransferDecoder cannot decode data after finishing") exceptions.RuntimeError: _IdentityTransferDecoder cannot decode data after finishing The code at filtering_proxy_listener.py:144 is keeping track of the received data size and then calls into the base class dataReceived(). Barry

1 0

[Twisted-Python] TB in tls.py
by Scott, Barry June 27, 2019

June 27, 2019

I'm see this TB often and I'm wondering how to fix it. File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 902, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/local/lib/python2.7/site-packages/twisted/web/http.py", line 2333, in forceAbortClient self.transport.abortConnection() File "/usr/local/lib/python2.7/site-packages/twisted/protocols/tls.py", line 435, in abortConnection self._shutdownTLS() File "/usr/local/lib/python2.7/site-packages/twisted/protocols/tls.py", line 338, in _shutdownTLS shutdownSuccess = self._tlsConnection.shutdown() exceptions.AttributeError: 'NoneType' object has no attribute 'shutdown' I can see in TLSMemoryBIOProtocol.connectionLost that _tlsConnection is set to None. Which I think is racing with the call later that calls forceAbortClient(). If that is the case then I think this is a possible fix? Thoughts? --- twisted-upstream/src/twisted/protocols/tls.py 2019-06-26 16:10:54.329822916 +0100 +++ _diff_.tmp 2019-06-26 16:12:17.992418082 +0100 @@ -334,7 +334,7 @@ Initiate, or reply to, the shutdown handshake of the TLS layer. """ try: - shutdownSuccess = self._tlsConnection.shutdown() + shutdownSuccess = self._tlsConnection is not None and self._tlsConnection.shutdown() except Error: # Mid-handshake, a call to shutdown() can result in a # WantWantReadError, or rather an SSL_ERR_WANT_READ; but pyOpenSSL Barry

2 1

[Twisted-Python] Running CPU bound function concurrently in twisted
by Chengi Liu June 25, 2019

June 25, 2019

Hi, I tried posting on SO, but havent been able to solve the problem. https://stackoverflow.com/questions/56696562/running-long-blocking-calculat… I have a very noob question. I have a function, which takes a second or two to process (CPU bounded) and then there is a for loop which runs that for loop.. Lets take this example def func(x): return x*x res = [func(i) for i in range(20)] Now, think of func being something complicated.. But the above is the idea. How do I run this function concurrently/ simultaneously using twisted framework. Thanks

5 10

[Twisted-Python] Azure Pipelines
by Jean-Paul Calderone June 23, 2019

June 23, 2019

Hello, I submitted ticket:9655 <https://twistedmatrix.com/trac/ticket/9655> / PR#1155 <https://github.com/twisted/twisted/pull/1155> for review a bit over a week ago. It is suffering from a red mark on the CI report due to Azure Pipelines complaining there was some "communication" problem "with the server". I imagine this is a spurious failure due to transient network conditions on the CI system. There doesn't appear to be a retry button to find out, though. I can't see any Azure Pipeline configuration in the repository, I don't see much information on the ticket that (maybe?) introduced Azure Pipeline usage <https://twistedmatrix.com/trac/ticket/9454>. I can't sign in to Azure or whatever the system driving this is called. What is to become of this PR? Thanks, Jean-Paul

4 10

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com June 23, 2019

June 23, 2019

1 0