Mailman 3 November 2015 - Twisted

[Twisted-Python] Twisted 15.5 Release Announcement
by Amber "Hawkie" Brown Nov. 29, 2015

Nov. 29, 2015

On behalf of Twisted Matrix Laboratories, I am honoured to announce the release of Twisted 15.5! The sixth (!!) release in 2015 has quite a few goodies in it -- incrementalism is the name of the game here, and everything is just a little better than it was before. Some of the highlights of this release are: - Python 3.5 support on POSIX was added, and Python 2.6 support was dropped. We also only support x64 Python on Windows 7 now. - More than nine additional modules have been ported to Python 3, ranging from Twisted Web's Agent and downloadPage, twisted.python.logfile, and many others, as well as... - twistd is ported to Python 3, and its first plugin, web, is ported. - twisted.python.url, a new URL/IRI abstraction, has been introduced to answer the question "just what IS a URL" in Twisted, once and for all. - NPN and ALPN support has been added to Twisted's TLS implementation, paving the way for HTTP/2. - Conch now supports the DH group14-sha1 and group-exchange-sha256 key exchange algorithms, as well as hmac-sha2-256 and hmac-sha2-512 MAC algorithms. Conch also works nicer with newer OpenSSH implementations. - Twisted's IRC support now has a sendCommand() method, which enables the use of sending messages with tags. - 55+ closed tickets overall. For more information, check the NEWS file (link provided below). You can find the downloads at <https://pypi.python.org/pypi/Twisted> (or alternatively <http://twistedmatrix.com/trac/wiki/Downloads>) . The NEWS file is also available at <https://github.com/twisted/twisted/blob/twisted-15.5.0/NEWS>. Also worth noting is the two Twisted Software Foundation fellows -- Adi Roiban and myself -- who have been able to dedicate time to reviewing tickets and generally pushing things along in the process. We're funded by the Twisted Software Foundation, which is, in turn, funded by donators and sponsors -- potentially like you! If you would like to know how you can assist in the continued funding of the Fellowship program, see our website: https://twistedmatrix.com/trac/wiki/TwistedSoftwareFoundation#BenefitsofSpo… Many thanks to everyone who had a part in this release - the supporters of the Twisted Software Foundation, the developers who contributed code as well as documentation, and all the people building great things with Twisted! Twisted Regards, Amber "Hawkie" Brown Twisted Release Manager, Twisted Fellow

2 1

[Twisted-Python] AMP message length limit
by Oon-Ee Ng Nov. 28, 2015

Nov. 28, 2015

I've just (to my surprise) hit this. As I understand from searching around, AMP messages are limited to ~64k due to the length prefix being 16-bit. A change in my internal data being sent (using dicts rather than lists) kicked one of my messages to way over that limit. There's a bit of discussion here - http://twistedmatrix.com/pipermail/twisted-python/2014-November/028947.html Is there an internal twisted solution planned, or should I go ahead and roll my own paging code? If the latter (as I strongly suspect), could I get some comments on this idea:- Original amp.Command had a single argument (amp.ListOf(amp.String()) and no response Modified amp.Command, 4 arguments and 1 response ID (sequentially generated by producer) - amp.Integer() CurPage - amp.Integer() TotalPage - amp.Integer() ActualData - amp.ListOf(amp.String()) Response - RecievedPage - amp.Integer() Questions:- 1. ID is so the client can be sure not to concatenate different lists 2. Do I need a response at all? 3. Should I attempt to plug as many list items as possible into each page (requires length checking of json-encoded strings and repeated encoding/checks) or just choose a suitable limit of list items (my current max length is about 200 characters and average is 71) of maybe 300 list items per message? My current list is about 1k items in all, and it's only going to get bigger. 4. I was intrigued by the mention of 'Tubes' in the link above. Found it here - https://tubes.readthedocs.org/en/latest/tube.html - should I be using that instead? I'm writing a homegrown app which will only really need (at this point) to communicate with itself and copies of itself, and settled with AMP as being a simple way of achieving that. Thanks for the time.

2 7

[Twisted-Python] doc breakage
by Phil Mayers Nov. 28, 2015

Nov. 28, 2015

Does this look right? https://twistedmatrix.com/documents/current/api/twisted.internet.interfaces… Looks like docs for logPrefix has stomped on docs for all the other interface methods?

4 4

[Twisted-Python] Question for obtaining mac-address
by Pavel Zverev Nov. 26, 2015

Nov. 26, 2015

Have nice day, developers! I am a beginner python developer, and recently I came across the Twisted framework, I was interested in whether I get the MAC-address of the client connected to the network, similar twisted.web.http.Request.getClientIP. Thanks.

2 1

[Twisted-Python] Twisted 15.5.0 Prerelease 2 Announcement
by Amber "Hawkie" Brown Nov. 25, 2015

Nov. 25, 2015

Hi everyone, Twisted 15.5.0pre2 is now available for download. This includes a bugfix for a regression in 15.5 which prevents Twisted Web's client from doing requests to raw IPv6 addresses. You can find the new release and the NEWS file at https://twistedmatrix.com/Releases/pre/15.5.0pre2/. Please test it! If no more issues are found, 15.5 will release sometime at the end of this week. Regards, Amber Brown Twisted Release Manager

1 0

[Twisted-Python] twisted/runner/inetdtap.py pyflakes warnings cleanup
by Adi Roiban Nov. 24, 2015

Nov. 24, 2015

Hi, Right now twisted/runner/inetdtap.py has the following errors reported by pyflakes: $ pyflakes twisted/runner/inetdtap.py twisted/runner/inetdtap.py:58: undefined name 'rpcVersions' twisted/runner/inetdtap.py:59: undefined name 'name' twisted/runner/inetdtap.py:62: undefined name 'p' twisted/runner/inetdtap.py:152: undefined name 'proto' >From what I can see from reading the code, RPCServer.startService is broken. makeService is also broken for rpc/* services . There are no tests for all this code. Link for the code : https://github.com/twisted/twisted/blob/trunk/twisted/runner/inetdtap.py ---------- I am working on getting the Twisted code clean of pyflakes errors/warnings so that contributors could run pyflakes check on their own branch, without relying on buildbot. I would like to clean the inetdtap.py module of pylakes warning but since it is broken and has no tests, I have no idea what this code is expected to do and how end users are expecting to use it. Does someone volunteer to fix it? Otherwise, has anyone any objections against deprecating twisted/runner/inetdtap.py? Since it is broken, can we just remove it? Thanks! -- Adi Roiban

2 3

[Twisted-Python] spambayes
by Paweł Miech Nov. 23, 2015

Nov. 23, 2015

I get following message: > Submission rejected as potential spam > SpamBayes determined spam probability of 74.83 SpamBayes seems to hate new users... This is after trying to submit following bug report: Executive summary page( http://twistedmatrix.com/documents/current/core/howto/vision.html) is probably first thing user will read when she visits Twisted page. At the moment this page is not providing important information about Twisted. What is missing and what should be present in introduction to Twisted: * explanation of difference between non-blocking and blocking IO * what is reactor, why is it important (mention of reactor design pattern) * introduction to core concepts of Twisted: protocol, factory, defferred etc It should be simple and easy to understand explanation that could give total noob overview of framework. I also think some paragraphs are misleading and not very informative, for example: > While Python by itself is a very powerful language, there are many facilities it lacks which other languages have spent great attention to adding. It sounds very vague and it is probably not true now isn't it? Python3 has its own async libraries in standard library so this sentence is misleading. > It began as a game, it is being used commercially in games, and it will be, I hope, an interactive and entertaining experience for the end-user. this could also appear misleading for newbie. New user will probably visit docs to learn more about networking, information about usage of twisted in games may not be relevant for most use cases. It should not be first thing user reads when she visits introduction. > As a platform, Twisted should be focused on integration. Ideally, all functionality will be accessible through all protocols. At this point user knows nothing about protocols and knows little about functionality of Twisted so this sentence will probably not mean much for newbie.

2 1

[Twisted-Python] Updating wincertstore for SSL certifcicates
by John Aherne Nov. 20, 2015

Nov. 20, 2015

Submission rejected as potential spam - SpamBayes determined spam probability of 99.60% Looks like the spam filter does not like me. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 2

[Twisted-Python] Updateing wincertstore for SSL certifcicates
by John Aherne Nov. 20, 2015

Nov. 20, 2015

Sorry this was meant to go to ticket #6371 not the wiki -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1

[Twisted-Python] Updating wincertstore for SSL certificates
by John Aherne Nov. 20, 2015

Nov. 20, 2015

I am posting here because my update on the wiki got rejected as spam. If I can get that fixed I will repost it to the wiki. Anyway here is what I wanted to post. Regards John Aherne Some additional information regarding wincertstore. My reading of the runes relating to automatic and dynamic updates is as follows: In an organisation with a sysadmin, dynamic updates will be turned off and an internal mechanism will be in place to update clients. Dynamic updates represents too much of a security risk. There is also an option from Microsoft on a daily basis to automatically update the certificate store So sysadmins can either arrange to have the daily update occur or have it downloaded to a central system under their control where they distribute the update plus any other certificates they need to add for company use. In smaller organisations no doubt the automatic update will be left in place. It would seem that the dynamic update is part of windows os and the application is not involved in the process. The verification either succeeds or fails. As an example see this link that contains this info:-- http://dreamlayers.blogspot.co.uk/2009/12/windows-7-cant-always-automatical… <<Today Secunia PSI refused to run with the message: "an error occurred while verifying the security certificate". Then I found that IE refused o show https://secunia.com because the certificate was "not issued by a trusted certificate authority". Firefox did not have a problem with that webpage. For some reason, IE did not recognize the "Thawte Server CA" certificate. IE also refused to recognize StartSSL. This was really weird, because as far as I know, Windows 7 is supposed to automatically update root certificates. Microsoft even explains how the process works in Vista. My first thought was that my firewall was blocking the update, but that was not it. Event log showed event 4100 from CAPI2, which is "Successful auto update retrieval of third-party root certificate". The problem was event 4110: "Failed to add certificate to Third-Party Root Certification Authorities store with error: A certificate chain could not be built to a trusted root authority." I manually downloaded and installed the latest root certificate update from Windows Update. After that, everything works. I'm just left wondering why I had to deal with this in the first place. A later comment says: I think this may be because Cryptographic Services (CryptSvc) was unable to access the Internet because of the firewall. Its description says that it includes the "Automatic Root Certificate Update Service, which retrieves root certificates from Windows Update". There are CAPI2 events relating to downloading and unpacking a root certificates .CAB file, and those do not appear in my event log.>> So windows cryptographic services does the dynamic check for certificate verification. However, you can always download the latest certificate store and update it yourself. This strikes me as similar to using the firefox download as happens on many linux systems and how the requests library works. I did a simple test to see if the dynamic update works on my win7 system. I used certmgr.msc to delete the GoDaddy root certificate. I then opened a browser to the godaddy site to see what would happen. In some side bar I saw a link that said the cert was failing but the main site site connected without problems. When I checked the certificate store there was still no godaddy root certificate. So what was happening? I then downloaded the update and updated my store. I ended up with about 200 entries. More than before but since I did not make a note I can't really be sure if I got more than when I started. I just need to run some more tests to see how this dynamic update is supposed to work. Manually updating the certificate store The link to manually update the certifcate store is below: To Manually install the certificates 1. Download http://download.windowsupdate.com/msdownload/update/v3/static/trustedr/en/r… 2. Extract the files using the command rootsupd.exe /c /t:C:\temp\extroot 3. from c:\temp\extroot run the following 4 commands (from an elevated prompt) updroots.exe authroots.sst updroots.exe updroots.sst updroots.exe -l roots.sst updroots.exe -d delroots.sst Here is an extract from microsoft technet explaining how the automatic update works. https://technet.microsoft.com/en-us/library/cc749331(WS.10).aspx <<The Update Root Certificates feature in Windows Vista is designed to automatically check the list of trusted authorities on the Windows Update Web site when this check is needed by a user's application. Specifically, if the application is presented with a certificate issued by a certification authority in a PKI that is not directly trusted, the Update Root Certificates feature (if it is not turned off) will contact the Windows Update Web site to see if Microsoft has added the certificate of the root CA to its list of trusted root certificates. If the CA has been added to the Microsoft list of trusted authorities, its certificate will automatically be added to the set of trusted root certificates on the user's computer. The Update Root Certificates feature can be turned off in Windows Vista by using Group Policy. For more information, see "Procedures for Viewing or Changing Group Policy Settings that Affect Certificates in Windows Vista," later in this section.>> Below is a link to a mozilla forum where the sysadmins have been complaining over the past 7 years that firefox will not support wincertstore so it can be integrated into their control systems. As a result they drop firefox and only support chrome and IE. https://bugzilla.mozilla.org/show_bug.cgi?id=432802 -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1