Mailman 3 July 2014 - Twisted

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 27 Jul '14

27 Jul '14

1 0

[Twisted-Python] EuroPython2014
by Jonas Brunsgaard 25 Jul '14

25 Jul '14

Any of your guys at Euro Python in Berlin, right now? If so, I would like to invite other Twisted users/dev for a beer later.. ;) -- Jonas Brunsgaard

2 2

[Twisted-Python] [ANN] txZMQ released
by Andrey Smirnov 21 Jul '14

21 Jul '14

Hi! I would like to announce txZMQ, ZeroMQ bindings for Twisted. txZMQ is based on pyzmq and requires recent ØMQ version (>=2.1). txZMQ uses ØMQ APIs to get file descriptor that is used to signal pending actions from ØMQ library IO thread running in separate thread. txZMQ should be usable with any Twisted reactor. ØMQ is: * The socket library that acts as a concurrency framework. * Carries messages across inproc, IPC, TCP, and multicast. * Connect N-to-N via fanout, pubsub, pipeline, request-reply. * Fast enough for clustered products and supercomputing. * Asynch I/O for scalable multicore message-passing apps. Bridging ØMQ and Twisted makes a nice match: fast simple messaging between Twisted instances solving complex problems :) Links: * PyPi: http://pypi.python.org/pypi/txZMQ/ * Source code: https://github.com/smira/pyzmq * pyzmq: http://pypi.python.org/pypi/pyzmq * ØMQ: http://www.zeromq.org/ -- Andrey Smirnov, Qik Web Team Lead

8 11

[Twisted-Python] ANN: txtorcon 0.10.1
by meejah 21 Jul '14

21 Jul '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, A new minor release of txtorcon exists, fixing a couple bugs introduced along with the endpoints feature in 0.10.0: * issue #78: Add tox tests and fix for Twisted 12.0.0 (and prior), as this is what Debian squeeze ships * issue #77: properly expand relative and tilde paths for hiddenServiceDir via endpoints * fix bug incorrectly issuing RuntimeError in brief window of time on event-listeners You can download the release from PyPI or GitHub (or of course "pip install txtorcon"): https://pypi.python.org/pypi/txtorcon/0.10.1 https://github.com/meejah/txtorcon/releases/tag/v0.10.1 sha256sum reports: 33f04523329b14accb2054b81c5da887c28b402c797ba895dc1ee58824e107f1 txtorcon-0.10.1-py2-none-any.whl 7a6e8fab71fd05c223d866b60b998cf308661ef1fc87d94e06c3b51f4ada4a6a txtorcon-0.10.1.tar.gz thanks, meejah -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iQEbBAEBAgAGBQJTzD25AAoJEMJgKAMSgGmnB8IH+MnRyCCKMgPJ2GjV01K+PVaU UgvUXhaqLdQUbViKeYy7KNst9d8Dm3ejHr21kedE8oeY/ztHlLdvtJtgWWvTOrTI qqh4wWfBVHeibSgsEzwNVdeJ3MtERPWuCrIkIWnathbVjfnkRW+cLPOtJUWtSi/d pdN4ZC+K1jBvSeHPCDhf8sXSqdzsOxXOWn/9SUaa70c7kMrbxjMxO1Jw/miqftMq /wo0vyXn4EeEdURa9hYFpUqgUbUFl6C1KiELNeHWtwGbGWku17bNgQn5HH7uY4/6 RODYBaAT327062m81ig7zQ6MmxKcngIG4Ic3qvJYeLerO/EnnNvzSs3Q+EGQeg== =nsEx -----END PGP SIGNATURE-----

1 0

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 20 Jul '14

20 Jul '14

1 0

[Twisted-Python] Bug in DeferredLock?
by Tapiwa Gutu 18 Jul '14

18 Jul '14

We recently started experiencing the following error: *RuntimeError: maximum recursion depth exceeded while calling a Python object* After a fair bit of debugging we eventually found the problem and implemented a fix for it in our codebase. We found that if a waiting queue on a DeferredLock is ~125 (depending on call depth) the error can error. I recommend releasing the lock in a callLater. The offending line is here: https://twistedmatrix.com/trac/browser/trunk/twisted/internet/defer.py#L1360 And the script below can be used to reproduce the issue. #!/usr/bin/env python from twisted.internet import defer, reactor LOCK = defer.DeferredLock() FIRST_RUN = True def deferred_sleep(delay): d = defer.Deferred() delay = reactor.callLater(delay, d.callback, True) d.delay = delay return d @defer.inlineCallbacks def synchronized_call(): global FIRST_RUN yield LOCK.acquire() try: if FIRST_RUN: FIRST_RUN = False yield deferred_sleep(2) print "Queue length: %s" % len(LOCK.waiting) finally: LOCK.release() #reactor.callLater(0, lambda: LOCK.release()) if __name__ == '__main__': count = 125 def start_test(): for i in xrange(count): synchronized_call() reactor.callWhenRunning(start_test) reactor.run() <http://www.ml.sun.ac.za/>

1 0

[Twisted-Python] twisted.names.client.getHostByName() vs. reactor.resolve()
by Benjamin Rutt 18 Jul '14

18 Jul '14

Hi, I need to do a DNS resolution of a hostname to an IPv4 address in order to then send out a UDP packet via twisted’s APIs. There appears to be 2 ways to do this. Is it better to use twisted.names.client.getHostByName(), or is it better to use reactor.resolve()? Is there a bias for one over the other? Is one deprecated and the other isn’t? BTW they appear to behave differently; at work, I have a hostname that the one (getHostByName) resolves to None in the callback, and the other (reactor.resolve) resolves to a valid IP address in the callback. That analysis is based on twisted 12.3.0 on linux. Naturally I'm preferring reactor.resolve at this point, as it works better for the hostname in question (it happens to be a hostname that uses a CNAME record to alias to another hostname). Thanks, -- Benjamin Rutt

3 2

[Twisted-Python] some competition and hopefully inspiration for conch
by Glyph Lefkowitz 17 Jul '14

17 Jul '14

There's a new Python-based asynchronous SSH client and server in town: <http://asyncssh.timeheart.net/#> This looks to have a much more straightforward API and way better documentation than Conch, although it's of course python 3.4+ only and lacks many of Conch's sophisticated (if somewhat idiosyncratic) terminal handling features. I hope that this project will spur a little friendly competition from Conch's maintainers, such as we are :-). (Just a thought, it would be great if <https://twistedmatrix.com/documents/current/conch/howto/index.html> looked more like that web page, in terms of how long the examples are, what is demonstrated, and what is documented...) -glyph

4 3

[Twisted-Python] dumb question on qt4reactor
by Kevin Mcintyre 15 Jul '14

15 Jul '14

calling reactor.callLater within qt4reactor in an attempt to async non webkit work but not seeing expected behavior. Webkit locks while the callLater function executes. Any insight appreciated. Should I move real work into separate reactor and "broker" calls into qt4reactor/webkit. I have xmlrpc and websocket hooks into browser...but looking for guidance as to most direct method of brokering calls between reactors. Any insight appreciated. Having a blast using QT browser as a back-end, see demo site www.athleets.com. See below for amazon S3 upload worker that I expected to async, but in qt4reactor does not appear to work as planned. from pyscewpt import fixed from pyscewpt.s3 import bucket_util import boto from boto.s3.bucket import Bucket from boto.s3.key import Key from twisted.internet import defer, reactor bucket_queue = defer.DeferredQueue() def upload_worker(data): print 'macth upload_worker', data save_match( data['league'], data['twitter'], data['local_file'], data['meta'], data['profile']) return len(bucket_queue.pending) def match_queue(ign=None): print 'match_queue:', len(bucket_queue.pending), len(bucket_queue.waiting) if ign is None or ign >= 2: print 'requeue:', len(bucket_queue.pending) , len(bucket_queue.waiting) d = bucket_queue.get() d.addCallback(upload_worker) d.addBoth(match_queue) return d else: return None def queue_save_match(queue_dic): bucket_queue.put(queue_dic) print 'queue_save_match', len(bucket_queue.pending), len(bucket_queue.waiting), queue_dic['twitter'] # if len(bucket_queue.pending) >= 5 and len(bucket_queue.waiting) == 0: print 'queue drop worker' reactor.callLater(0,match_queue,None) def save_match(league, twitter, systemfile, meta, profile): print 'save_match:', twitter, profile s3_filename = league + '/' + twitter + '.png' bucket_util.save_s3( boto.connect_s3().get_bucket('~bucket'), s3_filename, None, systemfile, 'image/png', 'public-read', meta + [('yes', False), ('no', False), ('maybe', False), ('profile', profile)] )

3 4

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 13 Jul '14

13 Jul '14

1 0