Mailman 3 October 2014 - Twisted

[Twisted-Python] Buildbot downtime
by Tom Prince 03 Mar '22

03 Mar '22

I am plannig to upgrade twisted's buildbot in the near future. In preperation for that, I am going to take down the production buildbot on Tuesday, June 26, around 1600 UTC[1]. During that time, the result of any builds done will be lost. Thus, I'd ask that people avoid committing to trunk during that time. I will send a reminder before I take down the buildbot both here, and on #twisted and #twisted-dev. Tom [1] http://timeanddate.com/worldclock/meetingtime.html?iso=20120626&p1=80&p2=22…

2 8

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika 01 May '19

01 May '19

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html>

8 10

[Twisted-Python] Better usage of Twisted on multi core processor
by Sumanth Puram 13 Mar '15

13 Mar '15

Hi all, I have a twisted TCP server to listens to client, processes requests, do mysql database operations if needed (using adbapi Connection pool) and return the result. Before deploying this in production, I want to know right way to configure the server. Since twisted is single threaded, how can I leverage multiple cores of my production machine (which has 6 cores and 16 GB RAM) ? One approach that I thought of was to start multiple instances of twisted server on different ports. This would help in using the other cores as well. What do you guys suggest ? Thanks -Sumanth

5 7

[Twisted-Python] Tubes!
by exarkun＠twistedmatrix.com 12 Dec '14

12 Dec '14

Hello all, Some of you may have heard rumors of some work in progress on a replacement for Twisted's IConsumer/IProducer interfaces. Tubes have been largely Glyph's effort (though a lot of people have contributed in one way or another). And a large effort it's been. Development is proceeding in a Twisted branch and comes to over three thousand lines of additions so far. Given the large size of the implementation and the long time that this effort has been underway (I remember the Twisted meetup at the Rackspace offices that *I* attended when I was visiting SF... a year and a half ago... at which point tubes wasn't exactly a brand new project) I'd like to re-raise the idea that the best next step for the project is to see some distribution in its *current* state. Specifically, I think it would be beneficial to set up a tubes project on Github under the Twisted organization and try for a release in the very near future. I think this has several advantages over the status quo: 1) As an independent project, tubes will attract more attention than it presently gets as a relatively unknown ticket & branch of Twisted. 2) As a separate Python package, the logistics of actually using tubes are simpler (just consider how you might declare a dependency on a branch of Twisted - keeping in mind you may want to use tubes in a project that already depends on some version of Twisted). It may not make sense to say that it is the same quality as Twisted proper right off the bat (on the other hand, it may well - I suspect tubes in its current form actually is a lot higher quality than large sections of Twisted) but that doesn't mean people (not to mention the tubes project) can't benefit from being able to experiment with it. 3) Decoupling tubes from Twisted frees tubes from certain of Twisted's policies which are more challenging to follow for the kind of non- trivial, brand new code base that tubes is. Technically we could just say that these policies don't apply to a tubes package *in* Twisted but this kind of subtle distinction is often lost on users (ie application developers). a) Twisted's compatibility policy need not apply. It could either be sped up or abandoned more thoroughly. I'm generally a fan of being backwards compatible even when you have few users because it actually makes development easier, but loosening the policy to say things might break if it's just really inconvenient to keep them working (whereas Twisted goes to the inconvenience to keep them working) seems reasonable. b) tubes can undergo a faster release cycle to benefit more from user feedback. 4) At this point, a normal review of the tubes branch is going to be a problem. We do not have good tools or mechanisms for dealing with branches this large. The code in the current tubes branch can just become master of a new project. Development going forward from this point should continue to follow the feature-branch, small-changes, pre- commit-peer-review process. But those 3k lines are written already. Short of an extremely expensive effort to break the work up into smaller, self-contained pieces there's simply never going to be a *good* review in the typical style. Additionally, it may turn out that tubes can remain independent indefinitely. Someday perhaps Twisted would come to depend on it to allow the various protocols and applications implemented in Twisted to benefit from the superior abstractions it provides. Or maybe once it has undergone a few iterations it will make sense to bring it back to Twisted. I don't think this needs to be decided now. There are downsides, of course. All of the boring maintenance involved with having a separate project - setting up CI, actually doing the releases, etc. Perhaps we could find some volunteers to help out with these tasks, though, in exchange for getting some great code out there? I'm curious what the folks out there who develop applications using Twisted would find to be the easiest path forward. I'm also curious to hear what Glyph thinks about all this. ;) Jean-Paul

10 14

[Twisted-Python] patch for t.p.logger.SyslogObserver (was #7549)
by Roberto Polli 05 Dec '14

05 Dec '14

Hi everybody, I found some time for adding a stub of SyslogObserver to the new t.p.logger. https://github.com/ioggstream/twisted/commit/991156e1124d61ad26da0be1212307… Feedback welcome! Peace, R. -- Roberto Polli Community Manager Babel - a business unit of Par-Tec S.p.A. - http://www.babel.it T: +39.06.9826.9651 M: +39.340.652.2736 F: +39.06.9826.9680 P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma) CONFIDENZIALE: Questo messaggio ed i suoi allegati sono di carattere confidenziale per i destinatari in indirizzo. E' vietato l'inoltro non autorizzato a destinatari diversi da quelli indicati nel messaggio originale. Se ricevuto per errore, l'uso del contenuto e' proibito; si prega di comunicarlo al mittente e cancellarlo immediatamente.

3 5

[Twisted-Python] porting twisted.spread to Python3
by Wolfgang Rohdewald 03 Nov '14

03 Nov '14

I now have a local git branch with about 70 commits, always rebased onto current trunk. Those test suites pass with Python2 and Python3: test_banana, test_jelly, test_pb, test_pbfailure, test_persisted I also have a real app using PB (kajongg, a KDE game). Running the server and the client with any combination of Python 2 and Python 3 works but more testing is needed there. Especially transferring failures with non-ascii test, so I need to inject random errors. spread.banana is now fully covered by tests (was at 85% before). There is still a lot of code in twisted.spread which has no tests yet. I do not think I want to add missing tests for everything, only for those regions where I do changes or where it is not too difficult. And of course if an application triggers a bug yet uncovered by tests. Can you point me to other PB applications I could use for testing? Preferrably ones that are already mostly compatible with PY3 themselves, but that is not a _must_, I would try porting them unless it turns out to be too much work. And preferrably ones developed by people with a non-ascii native language. I do not yet trust Failures and logging with non-ascii. Below is the current list of commits. Everything you see there is still much in flux. This includes shuffling code between commits, combining and splitting commits, changing order. I want to open one ticket per commit. Simple things hopefully not needing much discussion first. Estimating one week per review/commit to trunk, this process will take more than one year. Often I can only add a new ticket after the previous one has been integrated because they depend on each other. So if it really seems to take that long, at some point I might just decide to put what I have to github and not care anymore. Which of course means I might have to distribute a patched twisted version with my application. I really want to leave Python2 behind, and only twisted is in the way. Latest versions of KDE do not have Python2 bindings anymore (KDE Frameworks 5). This log of course includes things being in tickets up for review. git log --pretty='format:%s' --reverse trunk..pb3 spread.banana: selecting an unsupported dialect now raises an Exception spread.banana now raises BananaError when receiving pb messages without pb being the selected dialect spread.banana: specific error message about unicode objects spread.banana: test coverage goes up from 85% to 100% test.test_banana: eliminate duplicate code twisted.python.reflect.safe_str on Python 3 converts utf-8 encoded bytes to clean str instead of "b'a'" test.test_{failure,banana,persisted,reflect}: consistently name all test methods test_* test.test_jelly: add tests for yet untested code spread.jelly: support the sets module only for Python 2 twisted.spread: Assume that Python always has unicode support enabled Ticket 7653. spread.jelly: deprecate unjellying instances spread.jelly can now transfer objects with new style classes cred.credentials: implements->implementer cred.checkers: implements->implementer test.test_pb: implements->implementer persisted.styles: add new tests persisted.styles.unpickleMethod: remove dead code spread.flavors: fix import paths, making them work with Python 3 too spread: Python 3: add from __future__ do not use argument tuple unpacking anymore, not available in Python 3 spread.publish and test.test_pb: replace file by open test.test_pb: use a constant for "BRAINS!" python.compat: get_im_func, get_im_self, get_im_class and use them in spread persisted.styles: simplify unpickleMethod persisted.styles: Port to Python 3 python.compat: for PY3, define long=int and xrange=range spread.banana: replace StringIO.StringIO by io.BytesIO python.compat.networkChar: New, and use it in spread.banana python.compat.networkString: improve Exception message spread.banana: change literals from "" to b"" where appropriate test.test_banana: fix a wrong assertion message test.test_banana: use itertools.chain test.test_banana: Python 3 does not know type long spread: Python 3 does not have sys.maxint spread.banana and test: Complete port to Python 3: handling of data spread.jelly: Unjellying instance is not supported with Python 3 spread.jelly: unify usage of the types module spread.jelly: consistently use _atom identifiers spread.jelly: complete port to Python 3 spread.pb, test.test_pbfailure: new syntax for except spread.pb: do not use argument tuple unpacking anymore, not available in Python 3 test.test_jelly now passes all tests with Python 3 cred.credentials.updateHash accepts unicode add ported modules to python/dist3.py spread.flavors: consistently use _atom identifiers spread.flavors: Python 3: eliminate __cmp__. Only support __eq__ and __ne__ spread: Use networkString/nativeString where appropriate spread.flavors: switch atoms from str to b"" test.test_pb: improve test_concurrentLogin: use a non-ascii password spread.pb: eliminate __cmp__, using FancyEqMixin test.test_pb: add comment to test_tooManyRefs: too slow spread.pb: introduce *_atom constants spread.pb: Python 3: use list() around some generators spread.pb.Broker: improve Exception message spread.pb and test: Python 3: xrange = range spread.pb and test: Python 3: use byte literals spread.pb: failure2Copyable: use _newInstance from jelly spread.pb: replace types.ListType with list spread.pb and test: use networkChar where appropriate test.test_pb: change from StringIO to io.BytesIO spread.pb: Python 3: make *_atom bytes python.logger.formatWithCall: support non-ascii strings. Why exactly does kajongg need this? python.compat: networkIdentifier, nativeIdentifier. Not yet used. SQUASH spread.pb: CopiedFailure now handles non-ascii strings. TEST MISSING spread.flavors: RemoteCopy supports non-ascii identifiers. TEST MISSING. Why does kajongg need this? python.reflect: _safeFormat must handle unicode python.util: new debug function stack(). REMOVE AGAIN python.compat: nativeIdentifier accepts non-unicode. NEEDED WHY? TEST CASE MISSING test.test_{banana,jelly,pb,pbfailure,persisted}: from __future__ test.test_pbfailure: make literal strings bytes test.test_pbfailure: replace StringIO with io.BytesIO spread.flavors: translate keys of received dict into nativeString. INCOMPLETE -- Wolfgang

4 8

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 26 Oct '14

26 Oct '14

1 0

[Twisted-Python] Installing into anaconda
by Topjetboy＠Y 23 Oct '14

23 Oct '14

Anaconda is a python distro. It comes in many flavors depending on the desired interpreter (2.7, 3.4, etc) how does one select and install the correct version of twisted?

1 0

[Twisted-Python] Announcing Eliot 0.5.0: Structured Logging as Storytelling
by Itamar Turner-Trauring 23 Oct '14

23 Oct '14

Eliot provides a structured logging and tracing system for Python that generates log messages describing a forest of nested actions. Actions start and eventually finish, successfully or not. Log messages thus tell a story: what happened and what caused it. Eliot has first class support for Twisted, including integration with Trial. Eliot is released by ClusterHQ [2], the Data People for Docker(tm), and is available at https://pypi.python.org/pypi/eliot [3]. Documentation is available at https://eliot.readthedocs.org [4]. Here's what your logs might look like before using Eliot: Going to validate http://example.com/index.html [1]. Started download attempted. Download succeeded! Missing <title> element in "/html/body". Bad HTML entity in "/html/body/p[2]". 2 validation errors found! After switching to Eliot you'll get a tree of messages with both message contents and causal relationships encoded in a structured format: * {"action_type": "validate_page", "action_status": "started", "url": "http://example.com/index.html [1]"} * {"action_type": "download", "action_status": "started"} * {"action_type": "download", "action_status": "succeeded"} * {"action_type": "validate_html", "action_status": "started"} * {"message_type": "validation_error", "error_type": "missing_title", "xpath": "/html/head"} * {"message_type": "validation_error", "error_type": "bad_entity", "xpath": "/html/body/p[2]"} * {"action_type": "validate_html", "action_status": "failed", "exception": "validator.ValidationFailed"} * {"action_type": "validate_page", "action_status": "failed", "exception": "validator.ValidationFailed"} What's New in 0.5.0: * Added support for Python 3.4. * Most public methods and functions now have underscore-based equivalents to the camel case versions, e.g. eliot.write_traceback and eliot.writeTraceback, for use in PEP 8 styled programs. Twisted-facing APIs and pyunit assertions do not provide these additional APIs, as camel-case is the native idiom. * eliot.to_file outputs log messages to a file. * Documented how to load Eliot logging into ElasticSearch via Logstash. * Documentation has been significantly reorganized. Links: ------ [1] http://example.com/index.html [2] https://docs.clusterhq.com [3] https://pypi.python.org/pypi/eliot [4] https://eliot.readthedocs.org

1 0

[Twisted-Python] adding twisted to anaconda
by topjetboy 23 Oct '14

23 Oct '14

Anaconda is a python distribution with many modules already installed. The whole distro installs to a separate root folder named 'anaconda'. How do i install twisted to this?

1 0

Re: [Twisted-Python] Serializing cftp and ssh sessions
by exarkun＠twistedmatrix.com 22 Oct '14

22 Oct '14

On 17 Oct, 08:50 am, axel.rau(a)chaos1.de wrote: > >Am 14.10.2014 um 17:07 schrieb Axel Rau <Axel.Rau(a)chaos1.de>: >>>We might need more to go on. Do you have a complete, minimal >>>example? (See http://sscce.org/) >>The attached zip archive >Did someone could spend the time to look at it? >Is my goal utopian? I can't tell if this represents a bug in Twisted or not. After a quick look through the code, I'm not really clear on why the program would ever exit (I didn't find the calls to `reactor.stop`, I guess they're hidden somewhere in conch "library" code), nor quite which code paths you expect to work and which you expect to fail. What is the expected behavior if I run test.py after editing it to point at a host I have access to? Is that the desired behavior? If there is a second case, what one edit can I make to observe that behavior? How does the behavior differ from what you're trying to achieve? Separately: * You should rewrite this code to not use twisted.conch.client.connect. That's some random library code no one ever expected application code to use. It's public, so you can use it if you want, but that's no promise it's any good. * Use twisted.conch.endpoints.SSHCommandClientEndpoint to perform `exec` requests * Consider pitching in to complete https://twistedmatrix.com/trac/ticket/6617 so you can use the new `SSHSubsystemClientEndpoint` API that ticket introduces to set up your SFTP connections. Jean-Paul

2 3

[Twisted-Python] Mysql and adbapi
by aft 20 Oct '14

20 Oct '14

Hi, I'm trying to run the following program : from twisted.enterprise import adbapi dbpool = adbapi.ConnectionPool("MySQLdb", "khalid_sultaln", "root", "mysql") def xmlValues() : return dbpool.runQuery("SELECt * from xml_vaules limit 1") def printResult(l): print type(l) print l[0][0] xmlValues().addCallback(printResult) But its not printing anything. What exactly i'm doing wrong?

2 2

[Twisted-Python] Call For Participation: a review rally before the race!
by Glyph Lefkowitz 19 Oct '14

19 Oct '14

On November 1, ClusterHQ is going to be hosting a Twisted sprint in Boston. One of the things that tends to put a damper on such events is the perpetual backlog of tickets in review, many of which have been there seemingly forever. I would like to propose that we - the whole Twisted community, but especially Twisted committers - give these sprinters a morale boost, by giving them a clean slate, review-queue wise. This will let them focus on developing cool new stuff, getting a release out, fixing gnarly old bugs, or whatever strikes their fancy, instead of just grinding through reviews the whole time, or feeling bad about leaving those reviews outstanding. Right now, we have 58 tickets in review. <https://twistedmatrix.com/trac/report/25> Here's my proposal: I am looking for at least 4 volunteers for a "review rally" starting on Monday, October 13. If each of 5 reviewers (the 4 volunteers, plus myself) can review 2 tickets per day for one week, until Sunday, October 19, we can clear 70 tickets, which should take care of the existing 58 tickets currently queued up, plus any that get resubmitted during that week. While some volunteers should be Twisted committers since we have plenty of pending contributions from non-committers that need review, there are at least 11 tickets in review from core contributors right now, so non-committer volunteers are equally appreciated! Of course, if we get 9 volunteers, then each reviewer only has to do one ticket per day, an even more tractable goal. And so on into the higher numbers :-). Who is with me?!?! -glyph P.S.: While you should of course do this out of its self-evident merit, I do plan to recognize participation in some way, at least with a commemorative page on twistedmatrix.com, and possibly also with something a little goofy like a Mozilla Open Badge :-).

9 19

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 19 Oct '14

19 Oct '14

1 0

[Twisted-Python] Serializing cftp and ssh sessions
by Axel Rau 17 Oct '14

17 Oct '14

I have adapted t.c.s.cftp and one of the ssh examples to transfer files to a remote host and than execute some commands there. Both has to be done for several remote hosts in sequence. Start of the ssh session has to wait until the cftp sessions are complete and after that has completed, the cftp session for the next host has to be started . . . The cftp part begins so: def distributeStuff(host, tasks): . . . doConnect(reactor, options, tasks) def doConnect(reactor, options, tasks): . . . conn.tasks = tasks vhk = verifyHostKey uao = SSHUserAuthClient(options['user'], options, conn) connect(host, port, options, vhk, uao).addErrback(_ebExit) class SSHConn(connection.SSHConnection): def serviceStarted(self): self.openChannel(SSHSession()) I played with inlineCallbacks but could not find a point where to place my yielded function. I tried to overwrite channelClosed in SSHConn: def channelClosed(self, channel): super(SSHConn, self).channelClosed(channel) if len(self.channels) == 0: print ‚[Last channel closed.]' # open ssh ssession: . . . but that gave an unhandled error: 2014-10-14 07:54:34+0200 [SSHService ssh-connection on SSHClientTransport,client] Unhandled Error Traceback (most recent call last): File "/usr/local/py_venv/p27/lib/python2.7/site-packages/twisted/internet/tcp.py", line 214, in doRead return self._dataReceived(data) File "/usr/local/py_venv/p27/lib/python2.7/site-packages/twisted/internet/tcp.py", line 220, in _dataReceived rval = self.protocol.dataReceived(data) File "/usr/local/py_venv/p27/lib/python2.7/site-packages/twisted/conch/ssh/transport.py", line 461, in dataReceived self.dispatchMessage(messageNum, packet[1:]) File "/usr/local/py_venv/p27/lib/python2.7/site-packages/twisted/conch/ssh/transport.py", line 483, in dispatchMessage messageNum, payload) --- <exception caught here> --- File "/usr/local/py_venv/p27/lib/python2.7/site-packages/twisted/python/log.py", line 88, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File „/usr/local/py_venv/p27/lib/python2.7/site-p (bt stopped here). Please advise! Thanks, Axel — PGP-Key:29E99DD6 ☀ +49 151 2300 9283 ☀ computing @ chaos claudius

3 3

[Twisted-Python] Twisted hangs when using pyapns
by Ni Wesley 16 Oct '14

16 Oct '14

Hi all, I don't know if here is the right place. I hit a problem with pyapns which is based on twisted. So, I get code from our parter and setup up env involving interacting with APNS for IOS pushing work. I hit the problem that twisted hangs after a while's idle, I mean, when I start twisted with command: twistd -r epoll web --class=pyapns.server.APNSServer --port=7077 Then, raise an notification to IOS device by pyapns( https://github.com/samuraisam/pyapns),it's OK. However, if I let the system alone for a while, then, new raised notifications are all failed, what's more, if now I kill twisted process, and restart with the above command again, push operations are successful again. I use python2.7 and twisted 14.0.0 on Redhat6.4. So, anyone has hit this problem? Thanks. Wesley

4 7

[Twisted-Python] Spambayes problem with trac
by Nick Williams 16 Oct '14

16 Oct '14

Is there a secret method to get trac to accept a new ticket? I can't get the spam recognition percentage below 85% for my bug report, which looks very much unlike any spam I've ever seen.

1 0

[Twisted-Python] spread: differences between remote_ and perspective_ needing an explanation
by Wolfgang Rohdewald 14 Oct '14

14 Oct '14

Hi, while trying to isolate a bug in my app, I found some strange differences between the implementations of remote_* and perspective_*. 1. in pb.Broker.serialize(), those attributes are set and restored: jellyMethod, jellyArgs, jellyKw. Within twisted Those attributes are not used anywhere. I grepped, and I ran the test suite without those. Is there some code outside of twisted using this? What for? Or is there a need for avoiding garbage collection? (which would need some explanation). My theory is that it was forgotten to remove them when doing svn commit 1448: Before, they were actually used in _NetJellier._jelly_instance, but no more after this commit. 2. Those implementations differ: remote_*: return broker.serialize(state, self.perspective) perspective_*: return broker.serialize(state, self, method, args, kw) view_*: return broker.serialize(state, self.perspective, method, args, kw) observe_*: return broker.serialize(state, None, method, args, kw) Can you please explain why remote_* is implemented differently? This goes back to before the first import to svn, so I cannot browse the history. 3. They all raise AttributeError for undefined method except remote_* which raises MethodNotFound. Both are not documented in the API. Could this be unified to MethodNotFound? My theory is that svn commit 13473 was incomplete. Commit message was: Raise NoSuchMethod instead of AttributeError when call is made to remote_ method that doesn't exist -- Wolfgang

2 3

Re: [Twisted-Python] Call For Participation: a review rally before the race!
by Justin Mazzola Paluska 13 Oct '14

13 Oct '14

On 10/09/2014 06:56 PM, Glyph Lefkowitz wrote: On November 1, ClusterHQ is going to be hosting a Twisted sprint in Boston. ... I am looking for at least 4 volunteers for a "review rally" starting on Monday, October 13. If each of 5 reviewers (the 4 volunteers, plus myself) can review 2 tickets per day for one week, until Sunday, October 19, we can clear 70 tickets, which should take care of the existing 58 tickets currently queued up, plus any that get resubmitted during that week. I'm curious if any Boston-area Twisted developers or reviewers are going to be going to the Boston Python User Group Meetup tomorrow [Tuesday] at MS NERD. I ask because I'd like to get into helping out with Twisted—maybe by joining the review rally—and it's always more fun to get started with others. Either way, I am interested in reviewing code, if there is a way for a long-time Twisted user, but list lurker, to get started reviewing code. --Justin

2 1

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 12 Oct '14

12 Oct '14

1 0

[Twisted-Python] INCOMPATIBLE CHANGE: twisted.python.threadpool - time to test!
by Glyph 11 Oct '14

11 Oct '14

Again as per <https://twistedmatrix.com/trac/wiki/CompatibilityPolicy#ProcedureforExcepti…>, in order to conclude this discussion, I need to provide Twisted users with a branch to test their code against. Well, now you have it - and as per the policy you have a week from the sending of this message to register any objections :). The ticket <https://twistedmatrix.com/trac/ticket/2673> is in review now, and the branch is <desynchronize-2673-2>. Many of the existing tests used some of the compatibility features that we discussed previously, so I just left the existing tests intact. However, if you need any more compatibility features, please let me know and I can implement them in the branch. Feel free to reply here, or comment there. -glyph

2 2

[Twisted-Python] client logic
by Benjamin Schollnick 10 Oct '14

10 Oct '14

I’m sorry if this is a duplicate? I sent this earlier, but I don’t see it in the archives? Folks, I’m writing an client using twisted, and I’m basing it off the Poetry client from http://krondo.com/wp-content/uploads/2009/08/twisted-intro.html. I am porting a previous pyserial based client, so I’m running into new territory here. And it’s working fine, there are two “issues" that I am running into. class WT_Protocol(Protocol): tag = '' def to_bytes(self, seq): """convert a sequence to a bytes type""" # # from pyserial.util if isinstance(seq, bytes): return seq elif isinstance(seq, bytearray): return bytes(seq) elif isinstance(seq, memoryview): return seq.tobytes() else: b = bytearray() for item in seq: b.append(item) # this one handles int and str for our emulation and ints for Python 3.x return bytes(b) def connectionMade(self): print "Reseting the network" reset = self.to_bytes(packets.network_reset() ) self.transport.write ( reset ) self.transport.write ( self.to_bytes(packets.Set_SiteCode(NodeID=255, SC1=0, SC2=0, SC3=0) )) self.transport.write ( self.to_bytes(packets.Set_Gain_Packet(NodeID=255, Gain=1) )) # high gain self.transport.write ( self.to_bytes(packets.Set_Protocol_No_Nulls (NetworkID=0, ReceiverID=0, NodeID=1) )) # high gain time.sleep (.5) Now the client works right now, but I’m concerned that I can’t figure out how to immediately read for a reply packet from the server. For example, when I send the Set_SiteCode packet, I may receive a reply. But how would that occur in the Twisted framework? In my testing the dataReceived function is not being called, so I can’t trap it there, plus I’m not sure how I would figure that logic. Can anyone suggest a method to do this? The second question, as far as I can tell, isn’t very solvable. I’d like to add a “Press Q to quit” type logic to the code. But I don’t see a way to do this easily in twisted? (platform unspecific, but it’s running Mac OS X / Unix right now). - Ben

2 2

[Twisted-Python] a review to speed up other reviews?
by Glyph 09 Oct '14

09 Oct '14

<https://twistedmatrix.com/trac/ticket/7633> is keeping one of the buildbots persistently red. It's in review. There's some discussion on the ticket, If anyone has some free time to review it, this would make other code-reviews go just a little bit faster, since reviewers wouldn't need to keep looking at the logs for that builder to make sure that the failures are the "expected" ones; all the builders should be consistently green except for the pesky windows XP one. -glyph

3 6

[Twisted-Python] Boston Area Twisted Sprint
by exarkun＠twistedmatrix.com 08 Oct '14

08 Oct '14

Hello all, I'm happy to invite you to a Twisted sprint on Saturday, November 1st in Cambridge, Massachusetts. ClusterHQ will be hosting up to twenty (20) Twisted sprinters at the Cambridge Innovation Center ("Coronet City" room) in Kendall Square from 10AM until 11PM. The sprint topics are attendee choice - hacking on any part of Twisted is welcome. A large part of the ClusterHQ team will be taking part in the sprint. ClusterHQ is developing Flocker, a data management tool for use with Docker written in Python and based on Twisted. Follow us on Github: https://github.com/clusterhq/flocker ClusterHQ will provide lunch and dinner (something like delivery pizza, comment on the meetup page if you'd like to suggest something!). Find details and chat about sprint topics at the Boston Twisted Python meetup page: http://www.meetup.com/Boston-Twisted-Python/ Early birds, feel free to contact me off-list about meeting up for coffee at around 9AM before the sprint starts. Space at CIC is limited. Please RSVP on the Meetup page! See you there, Jean-Paul

1 0

[Twisted-Python] naming question before we end up committed...
by Glyph 08 Oct '14

08 Oct '14

While working on the package I just brought up in another thread, I realize we have another big new package coming in 14.1, "twisted.python.logger", and I'm wondering what the ".python" part of that package name is doing in there, and what value it adds. It's where "twisted.python.log" used to be, of course, but now I'm just wondering if we're just making the same mistake twice. Does anyone have an opinion on renaming it to "twisted.logger"? Or have a better suggestion? -glyph

5 5

[Twisted-Python] Client Logic
by Benjamin Schollnick 08 Oct '14

08 Oct '14

Folks, I’m writing an client using twisted, and I’m basing it off the Poetry client from http://krondo.com/wp-content/uploads/2009/08/twisted-intro.html. I am porting a previous pyserial based client, so I’m running into new territory here. And it’s working fine, there are two “issues" that I am running into. class WT_Protocol(Protocol): tag = '' def to_bytes(self, seq): """convert a sequence to a bytes type""" # # from pyserial.util if isinstance(seq, bytes): return seq elif isinstance(seq, bytearray): return bytes(seq) elif isinstance(seq, memoryview): return seq.tobytes() else: b = bytearray() for item in seq: b.append(item) # this one handles int and str for our emulation and ints for Python 3.x return bytes(b) def connectionMade(self): print "Reseting the network" reset = self.to_bytes(packets.network_reset() ) self.transport.write ( reset ) self.transport.write ( self.to_bytes(packets.Set_SiteCode(NodeID=255, SC1=0, SC2=0, SC3=0) )) self.transport.write ( self.to_bytes(packets.Set_Gain_Packet(NodeID=255, Gain=1) )) # high gain self.transport.write ( self.to_bytes(packets.Set_Protocol_No_Nulls (NetworkID=0, ReceiverID=0, NodeID=1) )) # high gain time.sleep (.5) Now the client works right now, but I’m concerned that I can’t figure out how to immediately read for a reply packet from the server. For example, when I send the Set_SiteCode packet, I may receive a reply. But how would that occur in the Twisted framework? In my testing the dataReceived function is not being called, so I can’t trap it there, plus I’m not sure how I would figure that logic. Can anyone suggest a method to do this? The second question, as far as I can tell, isn’t very solvable. I’d like to add a “Press Q to quit” type logic to the code. But I don’t see a way to do this easily in twisted? (platform unspecific, but it’s running Mac OS X / Unix right now). - Ben

2 1

[Twisted-Python] from: Cameron Dale
by Cameron Dale 07 Oct '14

07 Oct '14

Hiya http://minakow.ru/iron.php?pull=aqddrmtvcs3630rwfzrcp camrdale(a)gmail.com

1 0

[Twisted-Python] my branches merged by you (itamar)
by Wolfgang Rohdewald 05 Oct '14

05 Oct '14

Thanks for fixing my mistakes! Based on your fixes I started a personal checklist, see below. Is that list correct so far? (it will not include things I already did right) I noticed that you also changed 'x' to b'x' in some places. I did not do that yet because it is not needed for those tickets, I wanted to keep them small and focussed. Instead I am planning tickets for just that: (after minimizing the number of places to be fixed) twisted.spread.banana and tests: literal strings become bytes where appropriate twisted.spread.jelly and tests: define *_atom consistently for all atoms, change them to bytes twisted.spread.pb and tests: literal strings become bytes where appropriate The current checklist: Line length max. 79 *.rst doc: One line per sentence, no length limit fully qualified names in doc and news, twisted.* no __ in Twisted. Everything in a test case is private anyway. if possible __execute__ (not __define__) only one assert per test document changed API like raising more or different exceptions use new print() formatting syntax Document and isolate test code using the private API of the module to be tested. test everything with PY2.6 too did twistedchecker add new warnings? news: most frequent form: Qualified now does ... news about spread belong into twisted/topfiles doc: not pb but PB Epydoc docstring ends with point break docstring lines not sooner than needed avoid the word “test” in test docstrings do not use the “I am” form in docstrings test docstring: describe what happens, including exceptions doc: avoid mentioning the private API -- Wolfgang

2 4

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 05 Oct '14

05 Oct '14

1 0

[Twisted-Python] 'module' object has no attribute 'optionsForClientTLS'
by Orbitus 03 Oct '14

03 Oct '14

I am trying to execute the check_server_certificate.py and I am getting this error… can you guys help me figure out whats going on? its the exact sample code on the website running on Ubuntu 14.04. Here is the error output: Traceback (most recent call last): File "check_server_certificate.py", line 28, in <module> task.react(main, sys.argv[1:]) File "/usr/lib/python2.7/dist-packages/twisted/internet/task.py", line 867, in react finished = main(_reactor, *argv) File "check_server_certificate.py", line 6, in main options = ssl.optionsForClientTLS(hostname=host.decode('utf-8')) AttributeError: 'module' object has no attribute 'optionsForClientTLS' Here is the code (check_server_certificate.py): from __future__ import print_function import sys from twisted.internet import defer, endpoints, protocol, ssl, task, error def main(reactor, host, port=443): options = ssl.optionsForClientTLS(hostname=host.decode('utf-8')) port = int(port) class ShowCertificate(protocol.Protocol): def connectionMade(self): self.transport.write(b"GET / HTTP/1.0\r\n\r\n") self.done = defer.Deferred() def dataReceived(self, data): certificate = ssl.Certificate(self.transport.getPeerCertificate()) print("OK:", certificate) self.transport.abortConnection() def connectionLost(self, reason): print("Lost.") if not reason.check(error.ConnectionClosed): print("BAD:", reason.value) self.done.callback(None) return endpoints.connectProtocol( endpoints.SSL4ClientEndpoint(reactor, host, port, options), ShowCertificate() ).addCallback(lambda protocol: protocol.done) task.react(main, sys.argv[1:]) Rudy Aramayo | Developer | OrbitusRobotics LLC Direct: 626.386.8329 | Email: orbitus(a)orbitusrobotics.com | Web: www.orbitusrobotics.com "This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to which they are addressed. If you have received this email in error please notify me (orbitus(a)orbitusrobotics.com). If you are not the named addressee you should not disseminate, distribute or copy this e-mail and should delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.”

2 1

[Twisted-Python] INCOMPATIBLE CHANGE: twisted.python.threadpool
by Glyph Lefkowitz 02 Oct '14

02 Oct '14

Hi all, (For those of you wondering about the subject, note that this message is sent in compliance with This email sent in compliance with <https://twistedmatrix.com/trac/wiki/CompatibilityPolicy#ProcedureforExcepti…>.) I've been trying to improve the reliability of the buildbots. (On that note, by the way, several builders are failing and more will be soon as security updates roll out, it would be great if someone could review <https://twistedmatrix.com/trac/ticket/7651> before the whole fleet turns red.) In so doing I (re-)discovered this bug: <https://twistedmatrix.com/trac/ticket/2673>. At first I just wanted to delete an intermittently failing test which appears to be nonsense. However, after some prodding from exarkun, I investigated and discovered that this very poorly-written test case does in fact illustrate a real problem with our current threadpool implementation which can result in deadlocks, hangs on exit, and other unpleasantness. In my use of Twisted I have encountered several "huh, this can't happen, maybe cosmic rays or something" bugs which might have been caused by this, so I would very much like to fix it. One reason that our threadpool implementation is problematic is that it has a somewhat complex internal design, lots of weird little features (context preservation, completion callbacks, workload estimation) all built at the same layer with no composition. I tried to figure out a way to salvage its current architecture and I could not reasonably do so. I have prototyped a completely new threadpool implementation ("twisted.threads") in a spike. It is definitely not ready for review, but I would definitely appreciate design-level feedback on the code right now. You can see the implementation here: <https://twistedmatrix.com/trac/browser/branches/desynchronize-2673/twisted/…> (or here: <https://github.com/twisted/twisted/tree/desynchronize-2673/twisted/threads> if that's more your speed) It's less code than the existing implementation and provides more useful features at the same time. It might, in fact, provide some of the threading primitives that we would need for a reactor-per-thread implementation. However, there is a very ugly implementation detail that prevents us from marching onwards to a glorious multithreading future: twisted.internet.interfaces.IReactorThreads.getThreadPool. The nominally "public" interface provided by its documented return type, the concrete (old-style!) class twisted.python.threadpool.ThreadPool, is ridiculously over-broad. Here are the features (attributes and methods) in its "public" interface: callInThread callInThreadWithCallback adjustPoolsize start stop q min max joined started name waiters threads working workers start startAWorker stopAWorker dumpStats __getstate__ __setstate__ threadFactory currentThread Here's what I would like its public interface to be, more or less what it's intended to be, based on the ways it's documented and used in Twisted and in the various projects I can see using it: callInThread callInThreadWithCallback adjustPoolsize start stop It would not be too much effort to also preserve, for legacy purposes: min max joined started name dumpStats However, I would REALLY like to delete these implementation details: workers threads working startAWorker q __getstate__ __setstate__ threadFactory currentThread Deleting these implementation details straight up would also make it such that anyone who inherited from the public class ThreadPool would not be able to override any of these things (oh my goodness, why would you do that, I really hope nobody has done that). The three ways I could proceed, in order of my own preference, are: Put the new implementation with entirely new interfaces into twisted.threads, put a compatibility shim into twisted.python.threadpool.ThreadPool that wraps those objects, provides my more minimal interface proposed above, and deletes 90% of its tests. Change IReactorThreads.getThreadPool to return a documented interface considerably more restricted than Compatibility implications: twisted.internet.interfaces.IReactorThreads's contract would be relaxed. Implementors of IReactorThreads would not see an impact unless they were talking to deleted parts of ThreadPool internally. ThreadPool's clients would have several methods removed, and subclassing would effectively not be possible any more (a bunch of overridable hooks would have been removed, and no replacements would be provided). However, I prefer this option because there are an extremely restrictive set of use-cases (basically: only logging, any behavioral changes would have been broken) where clients could have made legitimate use of these attributes or overriding any of these functions. Delete all of twisted.python.threadpool's test coverage, deprecate the entire module, and delete it in the next release (14.1+1). This has the advantage of making the test suite reliable, and gets the benefits for any users of callInThread, but does not relay any of the benefits to users of ThreadPool. The only ones doing this directly in Twisted are deferToThreadPool (and by extension, deferToThread) and WSGIResource. I could update these in tandem, however; the code changes would be very small. I don't think it would be incompatible to make WSGIResource itself accept a different type of constructor argument (although if we don't, I wonder if subclassing anything with a constructor can be considered applicable to the compatibility policy in Twisted; hmm). Actually go through the trouble of emulating all the attributes on ThreadPool and emulating them as best I can. Delete the direct tests for ThreadPool itself and write a private subclass that will be returned from getThreadPool that returns a compatibility shim which isinstance(ThreadPool) for compatibility and still has all the gross attributes. This change would be technically compatible, but would be a huge amount of additional work and I am doubtful that it would provide any value. So, does anyone out there have any code which makes use of the aforementioned bad attributes of ThreadPool, whose applications would break if I removed them? If so, how can I make you feel as bad about yourselves for using it as I feel bad about myself for writing it? ;-) -glyph

7 10