Mailman 3 August 2016 - Twisted

[Twisted-Python] making appveyor a required status
by Glyph Lefkowitz Aug. 17, 2016

Aug. 17, 2016

Just a bit of Github housekeeping: Appveyor has looked pretty stable (more than our other Windows builds, even). I'm thinking about making it a required status for merging. But I'm curious what other folks have seen on PRs I might not have looked at; is it pretty reliable at this point? Also: I am thinking of turning off push statuses for Travis (for every branch but trunk), and just making it PR statuses. Is anyone using the push status, or is it just making builds take twice as long? -glyph

2 2

[Twisted-Python] [BACKWARDS INCOMPATIBILITY] Making twisted.python.dist3 private
by Amber "Hawkie" Brown Aug. 16, 2016

Aug. 16, 2016

Hi everyone, I am moving the dist3 from a whitelist to a blacklist, and that requires changing the whole module's interface, which shouldn't be public anyway. You can get the branch here: https://github.com/twisted/twisted/pull/501 , it'll be available for a further week. The only concern for this code was depgraph, which we are now at the point of not needing, as we're getting to leaf modules and not core modules. - Amber

3 2

[Twisted-Python] reading multipart/form-data headers
by Burak Arslan Aug. 16, 2016

Aug. 16, 2016

Hello All, You can find a sample HTTP POST request using HTTP multipart/form-data at the end of this message. The server that handles this request is using twisted so I end up with a Request object. Is there a way I can extract the file name ("image008.jpg") from this stream? I'm looking at the source of cgi.parse_multipart() and it seems to be ignored. Best regards Burak PS: POST /put HTTP/1.1 Host: localhost:7111 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/48.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: keep-alive Content-Type: multipart/form-data; boundary=---------------------------352471062160373366296932264 Content-Length: 382691 -----------------------------352471062160373366296932264 Content-Disposition: form-data; name="name" a -----------------------------352471062160373366296932264 Content-Disposition: form-data; name="version" 1 -----------------------------352471062160373366296932264 Content-Disposition: form-data; name="data"; filename="image008.jpg" Content-Type: image/jpeg (...)

3 4

[Twisted-Python] Twisted 16.3.1 Release Announcement
by Amber "Hawkie" Brown Aug. 15, 2016

Aug. 15, 2016

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 16.3.1. This is a bug fix & security fix release, and is recommended for all users of Twisted. The fixes are: - A bugfix for a HTTP/2 edge case, - Fix for CVE-2008-7317 (generating potentially guessable HTTP session identifiers) - Fix for CVE-2008-7318 (sending secure session cookies over insecured connections) - Fix for CVE-2016-1000111 (http://httpoxy.org/ <http://httpoxy.org/>) For more information, check the NEWS file (link provided below). You can find the downloads at <https://pypi.python.org/pypi/Twisted <https://pypi.python.org/pypi/Twisted>> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads <http://twistedmatrix.com/trac/wiki/Downloads>>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-16.3.1/NEWS <https://github.com/twisted/twisted/blob/twisted-16.3.1/NEWS>>. Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted! Twisted Regards, Amber Brown (HawkOwl)

1 0

[Twisted-Python] Twisted 16.3.1 Prerelease 1 Announcement
by Amber "Hawkie" Brown Aug. 15, 2016

Aug. 15, 2016

Hi everyone, I've just uploaded the first prerelease of Twisted 16.3.1, a security & critical bug fix release of the 16.3 series. It contains: - A bugfix for a HTTP/2 edge case, - Fix for CVE-2008-7317 (generating potentially guessable HTTP session identifiers) - Fix for CVE-2008-7318 (sending secure session cookies over insecured connections) - Fix for CVE-2016-1000111 (http://httpoxy.org/) You can find the tarball and the full news file for testing at https://twistedmatrix.com/Releases/pre/16.3.1pre1/ . If no issues are found, I will issue a full release by Friday. - Amber

2 1

[Twisted-Python] Twisted 16.4.0 Prerelease 1 Announcement
by Amber "Hawkie" Brown Aug. 15, 2016

Aug. 15, 2016

In a rare Twisted release double feature right after 16.3.1, I bring you the first prerelease of Twisted 16.4.0. This is a big release, the first after the Git migration has really kicked in! And so, here's what it brings us: - twist, a new command line tool for running Twisted plugins, similar to twistd but with a simpler, cleaner interface. - A new interface for Protocols, IHandshakeListener, which tells Twisted to tell the Protocol when the TLS handshake has been completed. - async/await support for Deferreds, allowing you to write Python 3.5+ coroutines using Twisted - Trial can be invoked with "python -m twisted.trial". - 35+ more modules ported to Python 3, and many many cleanups on the way to Python 3 on Windows support. - 240+ closed tickets overall. For more information, check the NEWS file (link provided below). As usual, it's available for download -- go here (https://twistedmatrix.com/Releases/pre/16.4.0pre1/ <https://twistedmatrix.com/Releases/pre/16.4.0pre1/>) to get the prerelease tarballs and the full NEWS file. If you want to install it right away, run: pip install https://twistedmatrix.com/Releases/pre/16.4.0pre1/Twisted-16.4.0rc1.tar.bz2 <https://twistedmatrix.com/Releases/pre/16.4.0pre1/Twisted-16.4.0rc1.tar.bz2> Please let me know if you have any issues, as well as if you don't! If everything works well, that's a good thing for me to know :) Twisted Regards, Amber Brown (HawkOwl)

6 7

[Twisted-Python] txtorcon 0.15.0
by meejah Aug. 11, 2016

Aug. 11, 2016

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm happy to announce txtorcon 0.15.0: * added support for NULL control-port-authentication which is often appropriate when used with a UNIX domain socket * switched to https://docs.python.org/3/library/ipaddress.html instead of Google's ipaddr; the API should be the same from a user perspective but **packagers and tutorials** will want to change their instructions slightly (``pip install ipaddress`` or ``apt-get install python-ipaddress`` are the new ways). * support the new ADD_ONION and DEL_ONION "ephemeral hidden services" commands in TorConfig * a first stealth-authentication implementation (for "normal" hidden services, not ephemeral) * bug-fix from https://github.com/david415 to raise ConnectionRefusedError instead of StopIteration when running out of SOCKS ports. * new feature from https://github.com/david415 adding a ``build_timeout_circuit`` method which provides a Deferred that callbacks only when the circuit is completely built and errbacks if the provided timeout expires. This is useful because :doc:`TorState.build_circuit` callbacks as soon as a Circuit instance can be provided (and then you'd use :doc:`Circuit.when_built` to find out when it's done building). * new feature from https://github.com/coffeemakr falling back to password authentication if cookie authentication isn't available (or fails, e.g. because the file isn't readable). * both TorState and TorConfig now have a ``.from_protocol`` class-method. * spec-compliant string-un-escaping from https://github.com/coffeemakr * fix https://github.com/meejah/txtorcon/issues/176 You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.15.0 https://github.com/meejah/txtorcon/releases/tag/v0.15.0 Releases are also available from the hidden service: http://timaq4ygg2iegci7.onion/txtorcon-0.15.0.tar.gz http://timaq4ygg2iegci7.onion/txtorcon-0.15.0.tar.gz.asc You can verify the sha256sum of both by running the following 4 lines in a shell wherever you have the files downloaded: cat <<EOF | sha256sum --check f2e8cdb130aa48d63c39603c2404d9496c669fa8b4c724497ca6bfa7752a9475 dist/txtorcon-0.15.0.tar.gz a359fb5e560263499400018262494378b3d347cd04a361adb08939df95ecedf6 dist/txtorcon-0.15.0-py2-none-any.whl EOF thanks, meejah -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJXl/KEAAoJEMJgKAMSgGmn76gH/1du7i9dmkMpr2PJrexVeXSo 9mSaeX/7KKaW71pEMmaCXfvhDJ6dMZDQpZ7saTM31zJZTp+MXjtHf0DZI2QTwgDw NYEBH+LO8PINN1ezPomgeZE6E4eJYlaDCyO6c7j3cOsEmohST+GPpvvdWdft+Sw2 hWvVf2+I4BV7vcIx6WQx4jKBS2gmlHbxuUv3LAnjj/Tn6oSYpft1IUK39pM66DX4 FzdYeBTloC6nzyH4sRTxnax+l9MfQJ2ZR+5alJi8uEvGlk580ciFASQNCVLaBY9r 4YALoipEg2Fm4BFA7qLsH0aFoLgx0lv7ng8lmpaP7XlPjUCuA7OcDp5jSqhGt2A= =hWml -----END PGP SIGNATURE-----

2 2

[Twisted-Python] clientfactory cleanup slow-down (after many http requests)
by Randomcoder Aug. 11, 2016

Aug. 11, 2016

Hello, I've been working on a small Twisted program. The program makes HTTP requests to a large number of feeds. Twisted is used to speed up the entire process. After the feeds are fetched, they're parsed. Finally they should be written to a database (to simplify the code, that part is left out). Feeds are fetched in parallel using gatherResults, and a batch is built. Then all batches are again gathered into a set of batches, a DeferredList is built out of those. A semaphore controls both the batch-level list of deferreds, and a semaphore controls the entire batch list deferred. Currently, the program works ok on 100-150 feeds, and BATCH_SIZE between 5 and 20. However, I notice the program starts to hang for a long time, when the number of feeds goes over 150-200. To be more precise, at the end of running the program, messages like these are printed, but the program seems to not be very active: Stopping factory <twisted.web.client._HTTP11ClientFactory instance at 0x7f0b7d5f3908> It seems like this is the cleanup phase. I've read what I could find on the topic. I wasn't able to make progress on it, so I'm posting to the mailing list to ask if someone has encountered this before. Maybe it's a common pitfall or issue that other people have also bumped into. Thanks

3 2

Re: [Twisted-Python] [Twisted] #7598: Port twisted.spread.pb to Python3
by Wolfgang Rohdewald Aug. 11, 2016

Aug. 11, 2016

Am Donnerstag, 28. Juli 2016, 11:27:15 CEST schrieb Twisted: > #7598: Port twisted.spread.pb to Python3 did you test with client on Python2 and server on Python3 and vice versa? Not a bug report, but anyway ... This was client with Python3, server with Python2 The same works with my old port to Python3: git clone https://github.com/wrohdewald/twisted.git git checkout spread-py3-7598 My version uses helpers in remoteMessageReceived: kw = nativeStringDict(broker.unserialize(kw)) method = getattr(self, "remote_%s" % nativeString(message), None) If you need a minimal example and an official bug report - that would take some more time. Maybe in a week or so. Peer will receive following PB traceback: Unhandled Error Traceback (most recent call last): File "/home/wr/src/kajongg/src/twisted/spread/banana.py", line 173, in gotItem self.callExpressionReceived(item) File "/home/wr/src/kajongg/src/twisted/spread/banana.py", line 136, in callExpressionReceived self.expressionReceived(obj) File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 575, in expressionReceived method(*sexp[1:]) File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 896, in proto_message self._recvMessage(self.localObjectForID, requestID, objectID, message, answerRequired, netArgs, netKw) --- <exception caught here> --- File "/home/wr/src/kajongg/src/twisted/spread/pb.py", line 913, in _recvMessage netResult = object.remoteMessageReceived(self, message, netArgs, netKw) File "/home/wr/src/kajongg/src/twisted/spread/flavors.py", line 120, in remoteMessageReceived state = method(*args, **kw) builtins.TypeError: remote_move() keywords must be strings -- Wolfgang

2 5

[Twisted-Python] twisted.internet.protocol.Protocol.connected flag purpose
by Adi Roiban Aug. 11, 2016

Aug. 11, 2016

Hi, Looking at the code in trunk [1] I can see that t.internet.protocol.Protocol will update the 'connected' attribute upon makeConnection but will not update it on connectionLost Is this the intended design? Is this design documented somewhere ? I have check the narrative docs [2] and api-docs[3] but this attribute is not documented. What is the purpose of t.internet.protocol.Protocol.connected ? Thanks! [1] https://github.com/twisted/twisted/blob/trunk/twisted/internet/protocol.py#… [2] http://twistedmatrix.com/documents/current/core/howto/servers.html#protocols [3] -- Adi Roiban

2 1