Mailman 3 November 2014 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince 03 Mar '22

03 Mar '22

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika 01 May '19

01 May '19

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html>

8 10

[Twisted-Python] Better usage of Twisted on multi core processor
by Sumanth Puram 13 Mar '15

13 Mar '15

Hi all, I have a twisted TCP server to listens to client, processes requests, do mysql database operations if needed (using adbapi Connection pool) and return the result. Before deploying this in production, I want to know right way to configure the server. Since twisted is single threaded, how can I leverage multiple cores of my production machine (which has 6 cores and 16 GB RAM) ? One approach that I thought of was to start multiple instances of twisted server on different ports. This would help in using the other cores as well. What do you guys suggest ? Thanks -Sumanth

5 7

[Twisted-Python] (no subject)
by Adi Roiban 07 Feb '15

07 Feb '15

As commented by @glyph Since we've added the buildbot link to trac pages, force-builds.py is basically obsolete. I think we should just scrub all mentions of it, and probably delete it. I have created a ticket to remove it and clean documentation: https://github.com/twisted/twisted-dev-tools/issues/5 Is there anyone else still using this tool? Should we keep it and rather than removing it, improve its documentation? Thanks! -- Adi Roiban

3 4

[Twisted-Python] Permissions to trigger buildbot tests
by Adi Roiban 22 Jan '15

22 Jan '15

Hi, To help with the review process I would like to ask permissions for triggering buildbot builds. Is there a wiki page describing what are the steps required for a regular contributor to get permissions to run buildbot tests and get write access to the repo? -------- In the same time, maybe we can write a few words about the steps required for a new contributor to become a full reviewer. Ex. 1. Contribute a few patches (ex 10) and learn the basic review process. Observe how reviewers respond to your patch. 2. Start doing review as junior reviewer, without merging. Once you are ok with the patch, invite another core developer to take a final view and merge the patch 3. Once you have reviewed a few patches without errors (ex 10) you can ask for full review permission or a core developer will let you know that you can merge the patch without asking someone else. This can be part of the current review process page: https://twistedmatrix.com/trac/wiki/ReviewProcess What do you think? Is not much, but should provide some guidance and hope! Thanks! -- Adi Roiban

5 23

[Twisted-Python] Tubes!
by exarkun＠twistedmatrix.com 12 Dec '14

12 Dec '14

Hello all, Some of you may have heard rumors of some work in progress on a replacement for Twisted's IConsumer/IProducer interfaces. Tubes have been largely Glyph's effort (though a lot of people have contributed in one way or another). And a large effort it's been. Development is proceeding in a Twisted branch and comes to over three thousand lines of additions so far. Given the large size of the implementation and the long time that this effort has been underway (I remember the Twisted meetup at the Rackspace offices that *I* attended when I was visiting SF... a year and a half ago... at which point tubes wasn't exactly a brand new project) I'd like to re-raise the idea that the best next step for the project is to see some distribution in its *current* state. Specifically, I think it would be beneficial to set up a tubes project on Github under the Twisted organization and try for a release in the very near future. I think this has several advantages over the status quo: 1) As an independent project, tubes will attract more attention than it presently gets as a relatively unknown ticket & branch of Twisted. 2) As a separate Python package, the logistics of actually using tubes are simpler (just consider how you might declare a dependency on a branch of Twisted - keeping in mind you may want to use tubes in a project that already depends on some version of Twisted). It may not make sense to say that it is the same quality as Twisted proper right off the bat (on the other hand, it may well - I suspect tubes in its current form actually is a lot higher quality than large sections of Twisted) but that doesn't mean people (not to mention the tubes project) can't benefit from being able to experiment with it. 3) Decoupling tubes from Twisted frees tubes from certain of Twisted's policies which are more challenging to follow for the kind of non- trivial, brand new code base that tubes is. Technically we could just say that these policies don't apply to a tubes package *in* Twisted but this kind of subtle distinction is often lost on users (ie application developers). a) Twisted's compatibility policy need not apply. It could either be sped up or abandoned more thoroughly. I'm generally a fan of being backwards compatible even when you have few users because it actually makes development easier, but loosening the policy to say things might break if it's just really inconvenient to keep them working (whereas Twisted goes to the inconvenience to keep them working) seems reasonable. b) tubes can undergo a faster release cycle to benefit more from user feedback. 4) At this point, a normal review of the tubes branch is going to be a problem. We do not have good tools or mechanisms for dealing with branches this large. The code in the current tubes branch can just become master of a new project. Development going forward from this point should continue to follow the feature-branch, small-changes, pre- commit-peer-review process. But those 3k lines are written already. Short of an extremely expensive effort to break the work up into smaller, self-contained pieces there's simply never going to be a *good* review in the typical style. Additionally, it may turn out that tubes can remain independent indefinitely. Someday perhaps Twisted would come to depend on it to allow the various protocols and applications implemented in Twisted to benefit from the superior abstractions it provides. Or maybe once it has undergone a few iterations it will make sense to bring it back to Twisted. I don't think this needs to be decided now. There are downsides, of course. All of the boring maintenance involved with having a separate project - setting up CI, actually doing the releases, etc. Perhaps we could find some volunteers to help out with these tasks, though, in exchange for getting some great code out there? I'm curious what the folks out there who develop applications using Twisted would find to be the easiest path forward. I'm also curious to hear what Glyph thinks about all this. ;) Jean-Paul

10 14

[Twisted-Python] patch for t.p.logger.SyslogObserver (was #7549)
by Roberto Polli 05 Dec '14

05 Dec '14

Hi everybody, I found some time for adding a stub of SyslogObserver to the new t.p.logger. https://github.com/ioggstream/twisted/commit/991156e1124d61ad26da0be1212307… Feedback welcome! Peace, R. -- Roberto Polli Community Manager Babel - a business unit of Par-Tec S.p.A. - http://www.babel.it T: +39.06.9826.9651 M: +39.340.652.2736 F: +39.06.9826.9680 P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma) CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale per i destinatari in indirizzo. E' vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore, l'uso del contenuto e' proibito; si prega di comunicarlo al mittente e cancellarlo immediatamente.

3 5

[Twisted-Python] doc bloat
by Glyph 04 Dec '14

04 Dec '14

Jean-Paul raised a salient point about documentation on a ticket <https://twistedmatrix.com/trac/ticket/4804#comment:21> recently. To quote that: We seem to be going the direction of "document every possible thing" these days. This stems from good intentions but the result is ever more bloated developer documentation of which any individual contributor has an ever shrinking knowledge. Rather than continuing to block the docs even further ... I think we need to get serious about pursuing a different strategy - for example, making twistedchecker a piece of software we could actually maintain and the output of which we could actually rely on (this really is just an example - the notion of a tool that tells you every single thing that's wrong with a piece of software is, of course, quite enticing - but perhaps unachievable). I think we've been moving in the direction of making twistedchecker maintainable, although it does still present some challenges. For example, I've been wanting to eliminate this <https://github.com/twisted/twistedchecker/issues/75 <https://github.com/twisted/twistedchecker/issues/75>> for a while, but I just haven't been able to figure it out. I am also thinking that we may want to create a category of private implementation details that don't require docstring coverage. In a public API, every parameter, every attribute, every detail should have accompanying prose and type annotations. In the innards of an implementation, these details can crowd out the code they're supposed to be elucidating. As a first approximation, I think we could ask twistedchecker to stop enforcing docstring requirements for objects directly matching a "private" naming pattern. Thoughts? -glyph

4 8

[Twisted-Python] HTTP combined log format in twisted.web.http changed from 13.2.0 to 14.0.0
by Free Ekanayaka 01 Dec '14

01 Dec '14

Hi, it was line = '%s - - %s "%s" %d %s "%s" "%s"\n' % ( in 13.2.0 (twisted/web/http.py line 1920), and it's u'"%(ip)s" - - %(timestamp)s "%(method)s %(uri)s %(protocol)s" ' so basically the client IP now gets wrapped within double quotes, e.g. a log line that was looking like: 1.2.3.4 - - [25/Oct/2004:12:31:59 +0000] "GET /dummy HTTP/1.0" 123 - "-" "-" it now looks like: "1.2.3.4" - - [25/Oct/2004:12:31:59 +0000] "GET /dummy HTTP/1.0" 123 - "-" "-" as one can see in the unit tests in test_web.py too. What's the reason for this change? It feels it can potentially break code that parses log files and it also seems to diverge from the format described on the Apache web site: http://httpd.apache.org/docs/1.3/logs.html Cheers, Free

1 1

[Twisted-Python] Importing/Exporting Putty and SSH.com keys
by Adi Roiban 30 Nov '14

30 Nov '14

Hi, I just want to let you know about a piece of code I was working on. I wanted to import and export Putty and SSH.com (Tectia) SSH keys in Python and I could not find any existing code for that... my final goal is to replicate the puttygen functionality. I have extended the current twisted.conch.ssh.key.Key class to import/export Putty and SSH.com keys Code is here: https://github.com/chevah/chevah-keycert Not sure if it make sense to have it integrated in Twisted. If there is someone willing to review (part of) and merge this code I am happy to send patches. Right now it depend on Twisted but only because of the original Key and packing/unpacking MP/NS values... Have a nice day! -- Adi Roiban

3 3

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 30 Nov '14

30 Nov '14

1 0

[Twisted-Python] I would like to port twisted to c++(14)
by Danny Shemesh 29 Nov '14

29 Nov '14

Hello everyone, I am Danny, a C++ dev, I'm familiar with twisted from past projects and by this day I am enchanted with its magic. I would like to port twisted to c++ (or at least the base ot it) and have couple of questions - 1. Is it legally ok to port it and open source it at github ? 2. Would anyone consider helping to this cause ? 3. Would you know of a code base that would benefit from such port ? Thanks!

4 3

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 23 Nov '14

23 Nov '14

1 0

[Twisted-Python] How do I debug this network problem?
by Peter Westlake 22 Nov '14

22 Nov '14

TL;DR - how do I debug the sequence of events between an AMP answer box arriving at a NIC, and AMP firing the callRemote Deferred? I have an application with two processes, on separate machines, communicating using AMP. One process does a callRemote, which returns a Deferred, which is never fired. I know from tcpdump that the AMP answer box arrives safely at the network interface card. This isn't something which can easily be reproduced. Instead, I want to ask the specific question: how do I debug the data path from the NIC to AMP firing its Deferred? I've had a look at the code, and got rather lost amongst the interfaces and inheritance and protocols and transports. If someone can help me narrow down the relevant bits of code, I can put in some Python tracing. FWIW, this is happening on Debian Squeeze and Wheezy, on VMs hosted on Xen 6.5. It only happens on some specific machines, and only sometimes. The same code has run flawlessly for many years elsewhere, though this same bug did happen there too some years ago. That time, it went away after most of the software in the system was upgraded. I tried that this time - Debian Squeeze to Wheezy, with associated kernel, Python and Twisted versions - but the problem persists. Anyway, I don't want to make the problem go away without understanding it, for fear that it will come back a third time. Peter.

3 22

[Twisted-Python] Twisted 14.0.2 & edns client subnet
by Jason J. W. Williams 22 Nov '14

22 Nov '14

Has anyone used the support for EDNS option information in the 14.0.2 twisted.names client to transmit EDNS client subnet information in upstream DNS queries? Any advice/pointers would be greatly appreciated. -J

1 0

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 16 Nov '14

16 Nov '14

1 0

[Twisted-Python] Sending longer messages in AMP
by Gavin Panella 14 Nov '14

14 Nov '14

Hi, We're using AMP and are starting to hit TooLong errors when scaling our application. In one respect it's a sign that we should do something like paging large requests and responses, but that's a lot more work, and comes with its own problems. We also don't need particularly large payloads: right now, a limit of ~500kiB would allow us to scale as far as we need and beyond. I've put together a fork of Twisted's AMP implementation that uses 32-bit length prefixes everywhere, though it limits the maximum message size to 2MiB. Every other aspect of it is the same so it's a drop-in replacement, as long as both ends of a connection use it. However, there's no negotiation phase so it's completely incompatible on the wire. The overhead of a few extra bytes is negligible for our use cases, where the networks are all assumed to be low-latency high-bandwidth LANs. Are there any reasons that we shouldn't be doing this? Was there a good reason for 16-bit length prefixes that still holds? Should we be doing something else? If I assume that the answers are all no, would someone find this protocol useful if we submitted it for inclusion in Twisted itself? The code right now is a straight copy of amp.py and test_amp.py with changes to 32-bit length prefixes everywhere, but for upstreaming we'd probably propose instead to modify the original to have an optional negotiation phase, and to make the maximum message size a parameter. Thanks! Gavin.

2 2

[Twisted-Python] is it possible to change the isolation level of a psycopg2 connection under enterprise.adbapi ?
by Jonathan Vanasco 13 Nov '14

13 Nov '14

there doesn't seem to be a way to access the connection objects within the pool ( psycopg2 manages this via `connection.set_isolation_level(X)` the only workaround I can think of seems to be emitting raw sql when I first start the transaction - but this doesn't seem right. am i missing anything?

2 1

[Twisted-Python] Manhole info needed
by Vikas Kumar 10 Nov '14

10 Nov '14

Hi, Can somebody tell me what is the need of manhole in twisted? Specially I've seen them in files like tap.py (twisted application process). One usecase I've seen them is wrt ShellFactory classes: manhole = ShellFactory() manhole.setService(self) internet.TCPServer(<someservice_name>.config.dget('jsonapi.port') + 10000, manhole, interface='127.0.0.1').setServiceParent(self) Could somebody please tell me what is being tried to achieve over here in the above code?

3 2

[Twisted-Python] twisted ssl server and client
by John Aherne 10 Nov '14

10 Nov '14

Hynek Well, thank you very much! Put it in and tested with request package and all seems to be working. Will test it later with the real system. That has saved me a lot of hassle. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1

[Twisted-Python] twisted ssl server and client
by John Aherne 10 Nov '14

10 Nov '14

<<You don’t need trustRoot for servers. You need to supply an extraCertChain which – as I’ve mentioned before – is easiest with the help of pem: https://github.com/hynek/pem#twisted>> Thanks. That link may be just what I need. I'll see how I get on. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

1 0

[Twisted-Python] twisted ssl server and client
by John Aherne 10 Nov '14

10 Nov '14

So I am now looking at CertificateOptions in more detail. But I am stuck trying to figure out how to add my GoDaddy cert to trustRoot. Here is what I am doing so far to see what is happening. Bu I think I am getting further and further from what I should be doing. If anyone can point out what must be the obvious thing to do, GREAT! #!/usr/bin/env python # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. import sys from twisted.internet import reactor, ssl, protocol, task, defer from twisted.python import log import echoserv log.startLogging(sys.stdout) ctx = ssl.CertificateOptions() certData2 = open('d:\\openssl\\ibook\\ibook2.pem', 'rb').read() certificate2 = ssl.PrivateCertificate.loadPEM(certData2) print 'CERT2', certificate2.options print 'CERT2', dir(certificate2) print 'CERT2 OPTIONS', certificate2.options() print 'CERT2OPTS', dir(certificate2.options) certGD = open('d:\\openssl\\ibook\\gd_bundle-g2-g1.crt', 'rb').read() #ctx.trustRoot = ssl.Certificate.loadPEM(certGD) certificate3 = certificate2.options(ssl.Certificate.loadPEM(certGD)) print 'cERT3', certificate3.privateKey print 'trustRoot2TROOT', certificate3.trustRoot #certificate2.options().trustRoot = ssl.Certificate.loadPEM(certGD) print 'trustRoot', certificate3 print 'trustRoot2type', type(certificate3.trustRoot) print 'trustRoot2TROOT', certificate3.trustRoot print 'trustRoot2PK', certificate3.privateKey print 'trustRoot2CERT', certificate3.certificate print 'trustRoot2VER', certificate3.verify #print 'trustRoot2', ctx.trustRoot factory = protocol.Factory.forProtocol(echoserv.Echo) reactor.listenSSL(8000, factory, certificate2.options(certificate3)) #reactor.listenSSL(8000, factory, certificate3) reactor.run() Thanks for any pointers. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1

[Twisted-Python] twisted ssl server and client
by John Aherne 10 Nov '14

10 Nov '14

Hynek Thanks for the information. Very interesting. I was wondering how to get the mozilla cert package. For the server I need to provide a certificate that can be verified by an external source. I used the requests package to test that it could verify my certificate without knowing exactly what it was doing. As far as I could see it verified the certificate and I then got an external source to also connect and send information having verified the certificate. That worked whereas before they had failed with SSL errors. So for the server, the certificate, key and godaddy bundle seem to be working. I found the DefaultOpenSSLContextFactory by googling for clues as to what to do. I then looked up the source to see what it required. And it seemed to do the trick. I have seen the documentation pointing at CertificateOptions but while I was trying to find out how I could get my godaddy bundle added in I found DefaultOpenSSlContextFactory which was easy to add my bundle to. It was not clear how I could do that with CertificateOptions. When I tried, I got unknown key argument because it seemed I could pass a key argument in but all my attempts failed. I have twisted 14.0.0 on one machine and twisted 14.0.2 on another and I bounce between them checking what works on one also works on the other. I have a lot of stuff running on the machine with 14.0.0 so I was not keen to upgrade until I had a clear idea of what I as doing. For the client I need to connect back to the external site. I can do this without verifying the certificate, but it would seem these days that is no longer sufficient. So I needed twisted.web.Agent to be able to verify the remote certificate. Trying that and once again googling for BrowserLikeContextFactory, I find that it has a real problem verifying hostnames. And your name features heavily in the list of people who have looked at the problem. I found that on windows 7 32 bit I could not get it to work. It always returned an SSL error. So for the time being I have abandoned that and am ignoring the problem. Just relying on finding and accepting any certificate. I need most of all to make sure that the server side is working properly, so your information is extremely welcome. I shall have to look at CertificateOptions again and see what the problem is. I have a feeling that everything is working with SSLv3 since all the error messages come back mentioning SSLv3. So I imagine at some point I need to be able to specify TLS and one of its variants. I hope I have explained a bit more clearly what I am trying to do. Regards -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

1 0

[Twisted-Python] twisted ssl server and client
by John Aherne 09 Nov '14

09 Nov '14

Thanks for the reply. In the end I took the examples in the docs and changed them to fit. So I have ended up with something that seems to work. But I wouldn't mind if someone can tell me if what I have done is miles wrong or spot on or could do with improvement or you have missed the point completely. Thanks for any pointers Below is the code with simple changes made #!/usr/bin/env python # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. echoserv_ssl.py import sys from twisted.internet import reactor, ssl, protocol, task, defer from twisted.python import log import echoserv log.startLogging(sys.stdout) #certificate = ssl.PrivateCertificate.loadPEM(certData) certificate2 = ssl.DefaultOpenSSLContextFactory('d:\\openssl\\ibook\\ibookservernp.key', 'd:\\openssl\\ibook\\7044921f82b7.crt') print 'CERT2', certificate2 my_context2 = certificate2.getContext() my_context2.load_verify_locations('d:\\openssl\\ibook\\gd_bundle-g2-g1.crt') factory = protocol.Factory.forProtocol(echoserv.Echo) print 'CERT OPTIONS', dir(certificate2) reactor.listenSSL(8000, factory, certificate2) echoserv.py #!/usr/bin/env python # Copyright (c) Twisted Matrix Laboratories. # See LICENSE for details. from twisted.internet.protocol import Protocol, Factory from twisted.internet import reactor ### Protocol Implementation # This is just about the simplest possible protocol class Echo(Protocol): def dataReceived(self, data): """ As soon as any data is received, write it back. """ print 'MESSAGE', data self.transport.write(data) #'HELLO THERE?\r\n') def main(): f = Factory() f.protocol = Echo reactor.listenTCP(8000, f) reactor.run() if __name__ == '__main__': main() -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 09 Nov '14

09 Nov '14

1 0

[Twisted-Python] twisted ssl server and client
by John Aherne 06 Nov '14

06 Nov '14

Apologies in advance for the rather basic questions I have here, but I am a bit stuck. I am looking at ssl with twisted 14.0.0 I have loaded all the dependencies crypto, pycrypto, service_identity, six, idna, cffi, pyasn1, pyopenssl 0.14, openssl 1.0.1g I am running this on windows7, windows 2008r2 I have read through Using TLS in Twisted several times but still find myself not sure as to what I should do. I need both server and client set up. I have a server end where I have a GoDaddy certificate and certificate chain. So on the server end I need to pass to ssl.CertificateOptions the privatekey and certificate. I also need to pass in the location of the GoDaddy bundle. I have a set up using Cherrypy and this now works fine. But I need it working with Twisted. Now this is where I am not so clear. Should I pass the bundle as TrustRoot or as extraCertChain. The docs seem to say that on Windows there is no cert store to be used as trustRoot. If anyone can throw some light on this I would be very grateful. For the client using Twisted.web.Agent, I need to verify a different ssl certificate on another server system I connect to. At the moment, I can connect without verifying the certificate, but these days that is no longer acceptable. So I try the example in the docs for checking a certificate but it fails on all examples, including www.twistedmatrix.com. I assume this is because I do not have a default set of certificates in a store to check against. If I use the requests package I can get it to verify or not the server certificate, but when I turn to twisted it is not clear what I should be doing. The example specifies an 'authority' public.pem for the client to check against. So should I be looking for the way requests works where I do not specify an authority and it works by finding a bunch of certs somewhere. Or do I pass in a cert that is specific to the server I am connecting to and it will only check against that. This seems more specific and more secure. A final point. I need to be able to confirm which version of openssl I am connecting with. Is there a way to pin down which version pyopenssl is finding. Thanks for any information and pointers. -- *John Aherne* *www.rocs.co.uk <http://www.rocs.co.uk>* 020 7223 7567

2 1

[Twisted-Python] The Path to Twisted 14.1
by HawkOwl 06 Nov '14

06 Nov '14

Hi everyone, To keep everyone in the loop... Since 14.0 came out months ago, and we really should have 14.1, I think it’s time we bit the bullet and cleaned out the regressions to get it out the door. Unfortunately, all of the currently open regressions are from the new logging system (#6750, plus #7548 and #7545) Since these regressions are introduced by a new ticket, we should revert them, and get the regressions fixed before it is remerged. I’ll make a new milestone called “New Twisted Logging” or similar so that they’re all still kept track of as a group. The only other big thing preventing 14.1 is https://twistedmatrix.com/trac/ticket/7647 , the patch that 14.0.1/14.0.2 was released to include. The patch fixes it, but not ideally, so it needs looking at. Once I’ve reverted those tickets and we can get #7647 refined and merged in some form, 14.1 will be out the door :) - hawkowl

4 10

[Twisted-Python] porting twisted.spread to Python3
by Wolfgang Rohdewald 04 Nov '14

04 Nov '14

I now have a local git branch with about 70 commits, always rebased onto current trunk. Those test suites pass with Python2 and Python3: test_banana, test_jelly, test_pb, test_pbfailure, test_persisted I also have a real app using PB (kajongg, a KDE game). Running the server and the client with any combination of Python 2 and Python 3 works but more testing is needed there. Especially transferring failures with non-ascii test, so I need to inject random errors. spread.banana is now fully covered by tests (was at 85% before). There is still a lot of code in twisted.spread which has no tests yet. I do not think I want to add missing tests for everything, only for those regions where I do changes or where it is not too difficult. And of course if an application triggers a bug yet uncovered by tests. Can you point me to other PB applications I could use for testing? Preferrably ones that are already mostly compatible with PY3 themselves, but that is not a _must_, I would try porting them unless it turns out to be too much work. And preferrably ones developed by people with a non-ascii native language. I do not yet trust Failures and logging with non-ascii. Below is the current list of commits. Everything you see there is still much in flux. This includes shuffling code between commits, combining and splitting commits, changing order. I want to open one ticket per commit. Simple things hopefully not needing much discussion first. Estimating one week per review/commit to trunk, this process will take more than one year. Often I can only add a new ticket after the previous one has been integrated because they depend on each other. So if it really seems to take that long, at some point I might just decide to put what I have to github and not care anymore. Which of course means I might have to distribute a patched twisted version with my application. I really want to leave Python2 behind, and only twisted is in the way. Latest versions of KDE do not have Python2 bindings anymore (KDE Frameworks 5). This log of course includes things being in tickets up for review. git log --pretty='format:%s' --reverse trunk..pb3 spread.banana: selecting an unsupported dialect now raises an Exception spread.banana now raises BananaError when receiving pb messages without pb being the selected dialect spread.banana: specific error message about unicode objects spread.banana: test coverage goes up from 85% to 100% test.test_banana: eliminate duplicate code twisted.python.reflect.safe_str on Python 3 converts utf-8 encoded bytes to clean str instead of "b'a'" test.test_{failure,banana,persisted,reflect}: consistently name all test methods test_* test.test_jelly: add tests for yet untested code spread.jelly: support the sets module only for Python 2 twisted.spread: Assume that Python always has unicode support enabled Ticket 7653. spread.jelly: deprecate unjellying instances spread.jelly can now transfer objects with new style classes cred.credentials: implements->implementer cred.checkers: implements->implementer test.test_pb: implements->implementer persisted.styles: add new tests persisted.styles.unpickleMethod: remove dead code spread.flavors: fix import paths, making them work with Python 3 too spread: Python 3: add from __future__ do not use argument tuple unpacking anymore, not available in Python 3 spread.publish and test.test_pb: replace file by open test.test_pb: use a constant for "BRAINS!" python.compat: get_im_func, get_im_self, get_im_class and use them in spread persisted.styles: simplify unpickleMethod persisted.styles: Port to Python 3 python.compat: for PY3, define long=int and xrange=range spread.banana: replace StringIO.StringIO by io.BytesIO python.compat.networkChar: New, and use it in spread.banana python.compat.networkString: improve Exception message spread.banana: change literals from "" to b"" where appropriate test.test_banana: fix a wrong assertion message test.test_banana: use itertools.chain test.test_banana: Python 3 does not know type long spread: Python 3 does not have sys.maxint spread.banana and test: Complete port to Python 3: handling of data spread.jelly: Unjellying instance is not supported with Python 3 spread.jelly: unify usage of the types module spread.jelly: consistently use _atom identifiers spread.jelly: complete port to Python 3 spread.pb, test.test_pbfailure: new syntax for except spread.pb: do not use argument tuple unpacking anymore, not available in Python 3 test.test_jelly now passes all tests with Python 3 cred.credentials.updateHash accepts unicode add ported modules to python/dist3.py spread.flavors: consistently use _atom identifiers spread.flavors: Python 3: eliminate __cmp__. Only support __eq__ and __ne__ spread: Use networkString/nativeString where appropriate spread.flavors: switch atoms from str to b"" test.test_pb: improve test_concurrentLogin: use a non-ascii password spread.pb: eliminate __cmp__, using FancyEqMixin test.test_pb: add comment to test_tooManyRefs: too slow spread.pb: introduce *_atom constants spread.pb: Python 3: use list() around some generators spread.pb.Broker: improve Exception message spread.pb and test: Python 3: xrange = range spread.pb and test: Python 3: use byte literals spread.pb: failure2Copyable: use _newInstance from jelly spread.pb: replace types.ListType with list spread.pb and test: use networkChar where appropriate test.test_pb: change from StringIO to io.BytesIO spread.pb: Python 3: make *_atom bytes python.logger.formatWithCall: support non-ascii strings. Why exactly does kajongg need this? python.compat: networkIdentifier, nativeIdentifier. Not yet used. SQUASH spread.pb: CopiedFailure now handles non-ascii strings. TEST MISSING spread.flavors: RemoteCopy supports non-ascii identifiers. TEST MISSING. Why does kajongg need this? python.reflect: _safeFormat must handle unicode python.util: new debug function stack(). REMOVE AGAIN python.compat: nativeIdentifier accepts non-unicode. NEEDED WHY? TEST CASE MISSING test.test_{banana,jelly,pb,pbfailure,persisted}: from __future__ test.test_pbfailure: make literal strings bytes test.test_pbfailure: replace StringIO with io.BytesIO spread.flavors: translate keys of received dict into nativeString. INCOMPLETE -- Wolfgang

4 8

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 02 Nov '14

02 Nov '14

1 0