Mailman 3 August 2016 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince March 3, 2022

March 3, 2022

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz Aug. 5, 2021

Aug. 5, 2021

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |… [View More]

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika May 1, 2019

May 1, 2019

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"… [View More]> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html> [View Less]

8 10

Re: [Twisted-Python] More tests running on Python 3
by Craig Rodrigues Nov. 14, 2016

Nov. 14, 2016

I looked at latest Travis builds on Linux: DATE BUILD ==== ===== August 11, 2016 https://travis-ci.org/twisted/twisted/jobs/151440581 Tests on Python 2.7 =============== 11640 (skips=2083, successes=9557) Tests on Python 3.5 ================ 9647 (skips=2008, successes=7639) -- Craig On Thu, Jun 30, 2016 at 3:43 PM, Craig Rodrigues <rodrigc(a)crodrigues.org <rodrigc(a)freebsd.org>> wrote: > Hi, > > I looked at … [View More]

3 3

[Twisted-Python] Twisted Challenge Coins!
by Ashwini Oruganti Sept. 30, 2016

Sept. 30, 2016

Hello! We are planning to get us some challenge coins, which is exciting! You should also be excited with us because this means: - we can now give it out to contributors at sprints and meetups. - challenge coins are awesome. Each coin costs about $6, and the minimum order is 100 coins. Glyph thinks everyone should have some to give out as tokens of appreciation to other contributors, and glyph's thoughts are usually worth following. So, if you are a Twisted contributor and would like to own … [View More]

8 10

[Twisted-Python] How do you determine the buffer size of a transport - a use-case for not using back pressure
by Steve Morin Sept. 28, 2016

Sept. 28, 2016

Twisted Community Problem: How do you determine the buffer size of a transport, to know how much data is waiting to be transmitted from using transport.write? Wait! You're going to say: use the Producer Consumer API ( http://twistedmatrix.com/documents/current/core/howto/producers.html ) To do what: So that instead of using back pressure I can check the buffer and when it's "too big/full" can decide to do something to the transport I am writing to: Slow transport handling options: - Buffer … [View More]to disk instead of memory - Kill the transport - Decide to skip sending some data - Send an error or message to the transport I am writing to - Reduce the resolution, increase the compression (things like video or audio) Why not use back pressure?: Some use-cases and some protocols this doesn't make sense. Examples: - You're sending video and if the receiver can't keep up you want to downgrade or upgrade the quality of the video, but if you don't know if you can't tell how much it buffering. - You're receiving from one connection and then broadcasting what you received to multiple clients and you want to handle it by sending an error to a client that doesn't keep up - You're sending a real-time protocol and want to skip sending some data that's no longer relevant if the buffer is too slow. On a server what are the consequences: Too much buffering in many transport write buffer can cause a server to fail. I don't know how to keep track of this to proactively without access to the buffer sizes. Resolutions can then be to, not accept new connections when memory pressure is too high, kill connections with the weakest/slowest clients or have a protocol that can have client switch connections to new servers when instructed to spread out the load. 1) I would like to hear on how people would solve this sort of problem in Twisted for a server? 2) I would like to hear people opinions on this in general. Tobias Oberstein - BCC'ed you on this email because you seems to have tackled similar problems (based on the mailing list) and would really love to get your take on this too. Glyph and Jean-Paul, you're also big on those threads so any opinions you have would be appreciated as well. Some of my background research * http://twistedmatrix.com/pipermail/twisted-python/2015-January/029064.html * Later but good in the chain: http://twistedmatrix.com/pipermail/twisted-python/2015-January/029071.html * Twisted receiving buffers swamped? * Summary: Great thread but runs into a tangent for a while and picks up good at the end again discussing producer again and the need for backpressure * Other: * http://crossbar.io/ * http://tavendo.com/ * Scenario: "Twisted is reading off the TCP stack from the kernel and buffering in userspace faster than the echo server is pushing out stuff to the TCP stack into the kernel. Hence, no TCP backpressure results, netperf happily sends more and more, and the memory of the Twisted process runs away." * Confirmed: Data isn't Buffered in "userspace inside Twisted, and before data is handled by the app in dataReceived." * https://twistedmatrix.com/pipermail/twisted-python/2010-September/022900.ht… * How to cap the buffering size of data to be sent in Protocol class * Summary: Same issue, very short no good info * https://twistedmatrix.com/pipermail/twisted-python/2012-March/025416.html * Limit on transport.write * Summary: Similar issue, very short no good info, glyph confirms that transport.write buffers everything sent to it. * https://twistedmatrix.com/pipermail/twisted-python/2012-February/025215.html * pushing out same message on 100k TCPs * Summary: Interesting discussion but different issue - interesting aside about irc spanning trees for a large broadcasts * http://twistedmatrix.com/pipermail/twisted-python/2008-April/017503.html * Question on push/pull producers inter-workings, was : "Is there a simple Producer/Consumer example or tutorial?" * Summary: Related and goes into what a producer is - an explanation of it * http://twistedmatrix.com/pipermail/twisted-python/2004-May/007732.html * When do calls to transport.write() block ? * Summary: Discusses the right issue, talks about buffer call back if full which would be great (if configurable) * https://pythonhosted.org/qbuf/qbuf.twisted_support.MultiBufferer-class.html * https://launchpad.net/qbuf * Summary: c buffer implementation that's incomplete thought might be promising * http://stackoverflow.com/questions/21821119/twisted-producer-to-many-client… * Summary: Was indicating a similar use-case but source doesn't seem to exist on the internet * http://twistedmatrix.com/documents/current/core/howto/producers.html * Summary: Documentation on Producer and Consumers but only help with a backpressure scenario * http://twistedmatrix.com/pipermail/twisted-python/2007-July/015690.html * Summary: Discussion on how producer-consumer API's work and future enhancement no help * MANY MORE THAT 100% NOT RELEVANT -- Steve Morin | Hacker, Entrepreneur, Startup Advisor twitter.com/SteveMorin | stevemorin.com *Live the dream start a startup. Make the world ... a better place.* [View Less]

5 4

[Twisted-Python] very weird 16.4 upgrade breakage
by Glyph Lefkowitz Aug. 31, 2016

Aug. 31, 2016

On Mimic, the requires.io PR that upgraded us to Twisted 16.4 broke the build. It did so in a really weird way: https://travis-ci.org/rackerlabs/mimic/jobs/156439316 Specifically, our mac app CI tests failed like so: File "pkg_resources/extern/__init__.pyc", line 61, in load_module exceptions.ImportError: The 'appdirs' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution. Any idea why 'appdirs' might … [View More]

1 0

[Twisted-Python] Twisted 16.4.0 Release Announcement
by Amber "Hawkie" Brown Aug. 28, 2016

Aug. 28, 2016

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 16.4.0. The highlights of this release are: - twist, a new command line tool for running Twisted plugins, similar to twistd but with a simpler, cleaner interface. - A new interface for Protocols, IHandshakeListener, which tells Twisted to tell the Protocol when the TLS handshake has been completed. - async/await support for Deferreds, allowing you to write Python 3.5+ coroutines using Twisted - Trial … [View More]

2 1

[Twisted-Python] Twisted 16.3.2/16.4.0 Release Errata
by Amber "Hawkie" Brown Aug. 28, 2016

Aug. 28, 2016

Hi everyone, Just a note to say that the HTTP/1 + TLS leaking CLOSE_WAIT sockets issue, fixed in 16.3.2 and 16.4.0, has been assigned a CVE, CVE-2016-7100. It'll start showing up on the various CVE databases soon. - Amber

1 0

[Twisted-Python] Twisted 16.4.1 Prerelease 1 Announcement
by Amber "Hawkie" Brown Aug. 28, 2016

Aug. 28, 2016

Hello again everyone, I bring you a prerelease of the Twisted 16.4.1 security fix release, thus deemed 16.4.1pre1. This release contains: - A mitigation for SWEET32 (https://www.openssl.org/blog/blog/2016/08/24/sweet32/ <https://www.openssl.org/blog/blog/2016/08/24/sweet32/>). - The inclusion of ChaCha20 to our TLS cipher suite, providing your underlying OpenSSL supports it. Please note that the SWEET32 mitigation will break TLS from IE on Windows XP. This shouldn't be an issue for you … [View More]

1 0

[Twisted-Python] Trial not operating on current working dir & Sunsetting Python 3.3
by Amber "Hawkie" Brown Aug. 27, 2016

Aug. 27, 2016

Hi everyone! Two things to let you all know in this email: 1. The new trial console script does not add the current directory to the PATH. Meaning, you can't use it to test a checkout of your code anymore! Instead, you should only use it for things that are actually on your path -- we recommend using something like tox to automate installing your package and testing your code. Hynek goes into detail here: https://hynek.me/articles/testing-packaging/ . If you really want to do this, you can … [View More]

1 0

[Twisted-Python] Waiting time for tests running on Travis CI and Buildbot
by Adi Roiban Aug. 24, 2016

Aug. 24, 2016

Hi, We now have 5 concurrent jobs on Travis-CI for the whole Twisted organization. If we want to reduce the waste of running push tests for a PR we should check that the other repos from the Twisted organization are doing the same. We now have in twisted/twisted 9 jobs per each build ... and for each push to a PR ... we run the tests for the push and for the PR merge... so those are 18 jobs for a commit. twisted/mantisa has 7 jobs per build, twisted/epsilon 3 jobs per build, twisted/nevow … [View More]14 jobs, twisted/axiom 6 jobs, twisted/txmongo 16 jobs .... so we are a bit over the limit of 5 jobs --------- I have asked Travis-CI how we can improve the waiting time for twisted/twisted jobs and for $6000 per year they can give us 15 concurrent jobs for the Twisted organization. This will not give us access to a faster waiting line for the OSX jobs. Also, I don't think that we can have twisted/twisted take priority inside the organization. If you think that we can raise $6000 per year for sponsoring our Travis-CI and that is worth increasing the queue size I can follow up with Travis-CI. ------------- I have also asked Circle CI for a free ride on their OSX builders, but it was put on hold as Glyph told me that Circe CI is slower than Travis. I have never used Circle CI. If you have a good experience with OSX on Circle CI I can continue the phone interview with Circle Ci so that we get the free access and see how it goes. -------- There are multiple ways in which we can improve the time a test takes to run on Travis-CI, but it will never be faster than buildbot with a slave which is always active and ready to start a job in 1 second and which already has 99% of the virtualev dependencies already installed. AFAIK the main concern with buildot, is that the slaves are always running so a malicious person could create a PR with some malware and then all our slaves will execute that malware. One way to mitigate this, is to use latent buildslaves and stop and reset a slave after each build, but this will also slow the build and lose the virtualenv ... which of docker based slave should not be a problem... but if we want Windows latent slaves it might increase the build time. What do you say if we protect our buildslaves with a firewall which only allows outgoing connections to buildmaster and github ... and have the slaves running only on RAX + Azure to simplify the firewall configuration? Will a malicious person still be interested of exploiting the slaves? I would be happy to help with buildbot configuration as I think that for TDD, buildbot_try with slaves which are always connected and virtualenv already created is the only acceptable CI system. Cheers -- Adi Roiban [View Less]

6 17

[Twisted-Python] Regarding moving on with ticket #7934 (New Logging System)
by Eeshan Garg Aug. 23, 2016

Aug. 23, 2016

Hello again! I am fully aware that it has been a _very_ long time since I mentioned that I would get on with <https://twistedmatrix.com/trac/ticket/7934>, but I haven't really had the time to actually do anything about it. :-/ I would love to work on more tickets like < https://twistedmatrix.com/trac/ticket/7897> and help port Twisted to the new logging system and kind of get back into the flow of submitting patches and contributing to Twisted again (I miss it so much!!). My main … [View More]

1 1

[Twisted-Python] Tests on Windows with Python 3 now passing
by Craig Rodrigues Aug. 23, 2016

Aug. 23, 2016

I have fixed the tests that were failing on Windows with Python 3. If you look at: https://ci.appveyor.com/project/adiroiban/twisted/build/trunk-1008 We now have builds running on Python 3.3, 3.4, and 3.5 on Windows. These builds are also uploading coverage results to codecov.io. I looked at that build and on Python 3.5 we have: 9539 tests (2338 skipped, 7201 succeeded) -- Craig

3 2

[Twisted-Python] Please fix the broken buildbots
by Craig Rodrigues Aug. 23, 2016

Aug. 23, 2016

Hi, Someone has made changes in the past few days which have broken many of the buildbots. For example, in this build: https://buildbot.twistedmatrix.com/builders/debian8-py2.7/builds/1548/steps… There is this: python -m tox -r -e py27-tests "" in dir /buildslave/debian8-py2.7/Twisted (timeout 1200 secs) watching logfiles {'test.log': 'build/py27-tests/tmp/_trial_temp/test.log'} argv: ['python', '-m', 'tox', '-r', '-e', 'py27-tests', ''] The last argument being passed to python is an … [View More]

5 7

[Twisted-Python] Twisted 16.4.0 Prerelease 2 Announcement
by Amber "Hawkie" Brown Aug. 21, 2016

Aug. 21, 2016

Hi everyone! I bring you a new Twisted 16.4.0 prerelease, with the following fixes: - Twisted's HTTP server, when operating over TLS, would not cleanly close sockets, causing it to build up CLOSE_WAIT sockets until it would eventually run out of file descriptors. (from 16.3.2) - AMP no longer fails on floats on Python 3. For more information, check the NEWS file (link provided below). As usual, it's available for download -- go here (https://twistedmatrix.com/Releases/pre/16.4.0pre2/ <… [View More]

1 0

[Twisted-Python] setup.py broken in trunk
by Glyph Lefkowitz Aug. 21, 2016

Aug. 21, 2016

41482c4fe5442bcc7b046ab53bf15409f2031abd makes a change such that 'pip install -e .' (and indeed, 'pip install .') no longer installs 'twist'. I think we should revert this. Clearly we have a problem with our test suite, if we're not catching the binaries disappearing on us :-). -glyph

2 3

[Twisted-Python] SSL4ClientEndpoint not updating the transport's connected flag on connection lost
by Adi Roiban Aug. 20, 2016

Aug. 20, 2016

Hi, I have observed that when a SSL4ClientEndpoint was used, the protocol's transport `connected` attribute is not updated by the wrapper when the connection is lost. I have this code which will need your own server cert+key file: https://gist.github.com/adiroiban/8e0be840f81750f88a587e634ddfb194 I tried it on twisted/trunk and when SSL is used the output is server:client connected client:connected client:send-message server:data received client:connection-lost client:transport-status:1:… [View More]

2 3

[Twisted-Python] Twisted 16.3.2 Release Announcement
by Amber "Hawkie" Brown Aug. 19, 2016

Aug. 19, 2016

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 16.3.1. This is a security fix release, and is recommended for all users of Twisted. The fixes contained are: - Twisted's HTTP server, when operating over TLS, would not cleanly close sockets, causing it to build up CLOSE_WAIT sockets until it would eventually run out of file descriptors. For more information, check the NEWS file (link provided below). You can find the downloads at <https://pypi.… [View More]

1 0

[Twisted-Python] Moving under a src directory, or: Sorry But I Broke All Your Pull Requests
by Amber "Hawkie" Brown Aug. 17, 2016

Aug. 17, 2016

Hi everyone, I have just merged the branch which puts twisted under a src/ directory. The reasons are many (see https://hynek.me/articles/testing-packaging/ for details as to why this is a Fantastic Idea), and it will require a bit of care with existing branches and pull requests. Thankfully, glyph has scripted all that your PR should need to do: python admin/fix-for-src-mv.py This will move everything under the src/ dir. If you have moved/renamed files in your PR, you may need to … [View More]

1 0

[Twisted-Python] making appveyor a required status
by Glyph Lefkowitz Aug. 17, 2016

Aug. 17, 2016

Just a bit of Github housekeeping: Appveyor has looked pretty stable (more than our other Windows builds, even). I'm thinking about making it a required status for merging. But I'm curious what other folks have seen on PRs I might not have looked at; is it pretty reliable at this point? Also: I am thinking of turning off push statuses for Travis (for every branch but trunk), and just making it PR statuses. Is anyone using the push status, or is it just making builds take twice as long? -glyph

2 2

[Twisted-Python] [BACKWARDS INCOMPATIBILITY] Making twisted.python.dist3 private
by Amber "Hawkie" Brown Aug. 16, 2016

Aug. 16, 2016

Hi everyone, I am moving the dist3 from a whitelist to a blacklist, and that requires changing the whole module's interface, which shouldn't be public anyway. You can get the branch here: https://github.com/twisted/twisted/pull/501 , it'll be available for a further week. The only concern for this code was depgraph, which we are now at the point of not needing, as we're getting to leaf modules and not core modules. - Amber

3 2

[Twisted-Python] reading multipart/form-data headers
by Burak Arslan Aug. 16, 2016

Aug. 16, 2016

Hello All, You can find a sample HTTP POST request using HTTP multipart/form-data at the end of this message. The server that handles this request is using twisted so I end up with a Request object. Is there a way I can extract the file name ("image008.jpg") from this stream? I'm looking at the source of cgi.parse_multipart() and it seems to be ignored. Best regards Burak PS: POST /put HTTP/1.1 Host: localhost:7111 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:48.0) Gecko/20100101 Firefox/… [View More]

3 4

[Twisted-Python] Twisted 16.3.1 Release Announcement
by Amber "Hawkie" Brown Aug. 15, 2016

Aug. 15, 2016

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 16.3.1. This is a bug fix & security fix release, and is recommended for all users of Twisted. The fixes are: - A bugfix for a HTTP/2 edge case, - Fix for CVE-2008-7317 (generating potentially guessable HTTP session identifiers) - Fix for CVE-2008-7318 (sending secure session cookies over insecured connections) - Fix for CVE-2016-1000111 (http://httpoxy.org/ <http://httpoxy.org/>) For more … [View More]

1 0

[Twisted-Python] Twisted 16.3.1 Prerelease 1 Announcement
by Amber "Hawkie" Brown Aug. 15, 2016

Aug. 15, 2016

Hi everyone, I've just uploaded the first prerelease of Twisted 16.3.1, a security & critical bug fix release of the 16.3 series. It contains: - A bugfix for a HTTP/2 edge case, - Fix for CVE-2008-7317 (generating potentially guessable HTTP session identifiers) - Fix for CVE-2008-7318 (sending secure session cookies over insecured connections) - Fix for CVE-2016-1000111 (http://httpoxy.org/) You can find the tarball and the full news file for testing at https://twistedmatrix.com/Releases/… [View More]

2 1