Mailman 3 July 2016 - Twisted

[Twisted-Python] removing twistedchecker buildbot for the time being
by Glyph Lefkowitz 20 Jul '16

20 Jul '16

Right now it seems the difference-computation logic on the twistedchecker buildbot has just broken completely. It's introducing useless noise into the build results because it makes every actually-passing build into a big red 'X' on the pull request status page. I think I'm going to remove it. Any objections? -glyph

2 2

[Twisted-Python] IProtocolWithReactor, or passing the reactor though to protocols
by Amber "Hawkie" Brown 18 Jul '16

18 Jul '16

(disclaimer: this is after several hours on an aeroplane, this may all be nonsense) So, I'm currently looking into adding some features to twisted.web -- something that historically hasn't had the best support for pluggable reactors, and which requires levels and levels of monkeypatching (see the top level of https://github.com/twisted/twisted/compare/trunk...deferreds-in-resrender-3… ). So, I was thinking, what if we made a new way for things to get their reactor, rather than extending all these concrete implementations everywhere? Currently, you'd pass a reactor instance through to protocols by making the factory understand it, and the protocol calling self.factory._reactor or whatever. I never thought this was a supremely helpful or useful interface -- especially for one-shot classes -- so what if instead we made a new IProtocol-extending interface, which signals to tcp.Port/etc that the protocol, once created, should have a reactor set on it (maybe through some "setConnectingReactor" or something). This would mean that the factory doesn't know about the reactor in most cases (with things like HTTPFactory doing logging as an exception) -- but we could also have an IClient/ServerFactoryWithReactor, that Endpoints (which needs to know about the reactor, and which reactor it is listening on) can then check for and call a similar function, telling the Factory what reactor it is actually running under. This, I think, would reduce a bunch of duplicate code in __init__s of factories, and could be implemented in the base class of Protocol or ServerFactory, possibly. What do people think? - Amber

3 7

[Twisted-Python] CI testing on Windows
by Adi Roiban 18 Jul '16

18 Jul '16

Hi, We now have automated tests running on Windows an Appveyor for any PR. In this way team and non-team member can get some feedback about their changes... and also see the coverage report for windows. Some tests are failing on the Appveyor env, while they pass on the the Win7 and Win2012 buildbot slaves. Here is the ticket https://twistedmatrix.com/trac/ticket/8612 Keeping buildbot running and Windows slaves running takes a lot of time. Coverage reports from Windows7 are broken... and I don't know why, while python processes on Win2012 are left running and some of them using up to 1.2G of memory. If you care about running Twisted on Windows please consider contributing to improve the Appveyor test or to manage the Windows slaves. I was the one pushing for the Win2012, but I don't have time to keep it running. If someone want to take care of it, please let me know. Otherwise I will remove it as I don't have time to monitor it and kill stray processes. I added Win2012 to get coverage reported from Windows, but now that we have Appveyor, the Win2012 slave is no longer that important. Many thanks to Rackspace for sponsoring the Win2012 slave and for Appveyor providing a great free service. Regards, -- Adi Roiban

2 1

[Twisted-Python] Trial running tests on Windows Python 3
by Craig Rodrigues 12 Jul '16

12 Jul '16

I have committed some fixes to trunk, and now trial is running tests on Windows Python 3. This buildbot is one example build with Python 3.5: https://buildbot.twistedmatrix.com/builders/win2012r2-64-py3.5/builds/12 There are some test failures, but a lot of things are passing. One nice thing to see in this build is that there are DeprecationWarnings where str is used when bytes should be used, and also ResourceWarnings for tests which are not cleanly shutting down sockets. -- Craig

1 0

[Twisted-Python] PyPy buildbot, less failed tests
by Craig Rodrigues 12 Jul '16

12 Jul '16

I would like to acknowledge the contributions of Ville Skyttä who submitted several patches to close open file descriptors in various Twisted tests. I looked at these two PyPy builds: https://buildbot.twistedmatrix.com/builders/ubuntu16.04-pypy5/builds/207 52 Failed tests (20 failures, 32 errors) https://buildbot.twistedmatrix.com/builders/ubuntu16.04-pypy5/builds/208 32 Failed tests (15 failures, 17 errors) This reduction is directly due to Ville's contributions. Thank you Ville!! -- Craig

1 0

[Twisted-Python] dropping old pyOpenSSL versions
by Glyph Lefkowitz 09 Jul '16

09 Jul '16

In the past, we've been very conservative about updating to require new versions of pyOpenSSL and cryptography. Right now we have a patch, <https://github.com/twisted/twisted/pull/146> (<https://twistedmatrix.com/trac/ticket/8441#comment:1>), that I'd like to just land. However, it establishes a dependency on a new version of pyOpenSSL, which transitively establishes a dependency on a new version of Cryptography. Generally, my thinking has evolved over the last few years to think that security dependencies like this should move fast, especially on projects (like pyOpenSSL and cryptography specifically) that don't maintain "stable" branches which do security patch-releases. In this specific case, the fix is not urgent; as it turns out, the netscape SPKI APIs actually do do the desired thing, which is just hashing the DER bytes of the key. (At the time I made the change to use Netscape SPKI, I thought it might be including somet other junk in the hash; we just lucked out here.) It's just a gross API for doing it which we should stop using now that better APIs have been exposed to do the same thing. However, it bears discussing - what are the things that hold us to older versions of pyOpenSSL and cryptography? Is there any good reason not to move our version pins forward whenever there's a new API or feature that we'd like, even for something simple like this cleanup? My default position is "upgrade upgrade upgrade" so if there's not a lot of interest in this discussion I'll probably just land the PR in question as-is. Thanks all, -glyph

4 8

[Twisted-Python] Twisted 16.3.0 Release Announcement
by Amber Hawkie Brown 06 Jul '16

06 Jul '16

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 16.3.0. The highlights of this release are: - The Git migration has happened, so we've updated our development documentation to match. We're now trialling accepting pull requests at github.com/twisted/twisted, so if you've ever wanted an excuse to contribute, now's the chance! - In our steady shedding of baggage, twisted.spread.ui, twisted.manhole (not to be confused with twisted.conch.manhole!), and a bunch of old and deprecated stuff from twisted.python.reflect and twisted.protocols.sip have been removed. - twisted.web's HTTP server now handles pipelined requests better -- it used to try and process them in parallel, but this was fraught with problems and now it processes them in series, which is less surprising to code that expects the Request's transport to not be buffered (e.g. WebSockets). There is also a bugfix for HTTP timeouts not working in 16.2. - Twisted now has HTTP/2 support in its web server! This is currently not available by default -- you will need to install hyper-h2, which is available in the [h2] setuptools extras. If you want to play around with it "pip install twisted[h2]" (on Python 2, a bugfix release will make it available on Python 3) and add "h2" to your Site's acceptableProtocols. - 53 tickets closed overall, including cleanups that move us closer to a total Python 3 port. For more information, check the NEWS file (link provided below). You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>). The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-16.3.0/NEWS>. Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted! Twisted Regards, Amber Brown (HawkOwl)

1 0

[Twisted-Python] Coverage exceptions
by Adi Roiban 06 Jul '16

06 Jul '16

Hi, Recently we have introduced a hard check of 100% coverage for all changes. This is done via coverage + codecov + github protected branches. Now, if your patch is not 100% covered github will not let you merge it. See for example this change: https://github.com/twisted/twisted/pull/261/files#diff-0fea8a8ca713deb7ea6a… The errback is there to help with test failures ... but the test should never fail, so that errback is never called... and that line is not covered. How should we proceed with these changes? Maybe this is not the best example and that code could be refactored... but I think that the topic of ignoring missing coverage is still valid. I suggest to introduce ` # pragma: no cover` and update the coverage config with [report] exclude_lines = pragma: no cover ----------- What do you think? Thanks! -- Adi Roiban

6 15

Re: [Twisted-Python] Buildbot upgrade to latest eight branch
by Adi Roiban 04 Jul '16

04 Jul '16

On 25 June 2016 at 16:53, Tom Prince <tom.prince(a)ualberta.net> wrote: > > https://github.com/twisted-infra/braid/pull/206 > > I'll try to review this this weekend. > I have merged the GitHub commit status as I think that it was working OK. I have now deployed the GitHub Hooks build trigger changes. Build are automatically triggered for all pushes in all branches. There is a delay of 5 seconds so that multiple builds pushed in the same time will only trigger a single build for the last commit. See this PR for details and if you want to review it :) https://github.com/twisted-infra/braid/pull/208 With this change, Buildbot should have a closer behavior to Travis-CI. That is, it will automatically start the builders and will report the build results in GitHub PR. Happy hacking :) -- Adi Roiban

2 2

[Twisted-Python] Python 3 code coverage not working?
by Craig Rodrigues 04 Jul '16

04 Jul '16

Hi, I submitted this pull request: https://github.com/twisted/twisted/pull/224 and codecov is saying that only 66.67% of the patch is covered because it is not hitting the Python 3 specific block: https://codecov.io/gh/twisted/twisted/compare/502f3a1e2cc125d214b6b7df5b173… This buildbot ran the build on Python 3 and uploaded a coverage report: https://buildbot.twistedmatrix.com/builders/fedora22-py3.4-coverage/builds/… Is something wrong here? I don't understand how Python 3 coverage is not reported. -- Craig

2 6