Mailman 3 October 2013 - Twisted

Re: [Twisted-Python] Flow-control for Pipes / Unix Domains Sockets?
by Tom Sheffler 23 Oct '13

23 Oct '13

Tobias and Itamar - Thanks for sharing the conversation and statistics. Very interesting! -Tom

1 0

[Twisted-Python] Flow-control for Pipes / Unix Domains Sockets?
by Tobias Oberstein 23 Oct '13

23 Oct '13

Hi, with TCP (either remote or loopback) I can have flow-control using the producer-consumer machinery that Twisted provides. Is that (flow-control / producer-consumer) also available (and practically usable/recommended) with: a) Unix Domain sockets b) Pipes (spawnProcess) ? Thanks! /Tobias

3 5

[Twisted-Python] Twisted as a full solution for web hosting [WSGI + other]
by Orestis Markou 23 Oct '13

23 Oct '13

Hello, Short form of the question: Are people using Twisted to host WSGI applications AND serve static files AND replace celery/redis/other? Are there any inherent drawbacks in using Twisted for this use case? Long form of the question: These days to get a reasonably feature-full python web stack deployed you need to have a lot of 3rd-party libraries. The bare minimum looks probably like so: 1. Your web-framework of choice, like Django 2. Some kind of WSGI container, like gunicorn 3. A static file server, like nginx 4. Some kind of database [off-topic for this message] Additionally, you might want: * Celery * Redis * Cron * Something for web sockets or similar technology * … and so on In my experience, Twisted can be used to replace a lot of those use cases: * It has a WSGI container * It has a static web server * It can be used for other long-running tasks I'd like to know if there is some kind of inherent drawback of using Twisted to fill those areas. My use case deals with many small intranet-like deployments of web applications, and I'd like to streamline the stack as much as possible. I believe that with the newly-released Crochet someone could even write a Django extension to replace `runserver` with something more production-oriented. Orestis

5 9

[Twisted-Python] Twisted 13.2.0pre1 Announcement
by HawkOwl 22 Oct '13

22 Oct '13

Hello everyone! I bring news of a Twisted pre-release - 13.2.0pre1, to be exact! This will be followed by the full release of 13.2.0 in the coming weeks. The highlights of this release are: * Twisted now includes a HostnameEndpoint implementation which uses IPv4 and IPv6 in parallel, speeding up the connection by using whichever connects first (the 'Happy Eyeballs'/RFC 6555 algorithm). (#4859) * Improved support for Cancellable Deferreds by kaizhang, our GSoC student. (#4320, #6532, #6572, #6639) * Improved Twisted.Mail documentation by shira, our Outreach Program for Women intern. (#6649, #6652) * twistd now waits for the application to start successfully before exiting after daemonization. (#823) * SSL server endpoint string descriptions now support the specification of chain certificates. (#6499) * Over 70 closed tickets since 13.1.0. For more information, check the NEWS file (link provided below). The pre-release tarballs can be found at the link below - please download them and test them as much as you can. Tarballs: http://twistedmatrix.com/Releases/pre/13.2.0pre1/ NEWS.txt: http://twistedmatrix.com/Releases/pre/13.2.0pre1/NEWS.txt Twisted Regards, HawkOwl

2 1

[Twisted-Python] serial communication - getting started
by Robert Voigtländer 22 Oct '13

22 Oct '13

Hi, I am new to Python. Trying to figure out how to use the serial port I started with pyserial. Works - but the CPU usage on my Raspberry Pi is at about 30% just for reading the port. Before I started to optimize the code it was even 90+%. So I tried twisted - and boom down to 3-4%. Even though twisted also uses pyserial (as far as I read) - obviously better than I do. So far so good. With copy&paste from the internet and some editing I got the PI to read from my Arduino. (code below) But the code seems to be blocking. I tried putting things in a thread - but I must have done something wrong. It is still blocking. I also don't know how to write. Does s.write("test") work? Would be great if someone could help me out here. Thanks a lot. Robert import logging from twisted.protocols.basic import LineReceiver from twisted.protocols.basic import LineReceiver from twisted.internet import reactor from twisted.internet.serialport import SerialPort from twisted.python import usage import thread class THOptions(usage.Options): optParameters = [ ['baudrate', 'b', 115200, 'Serial baudrate'], ['port', 'p', '/dev/ttyACM0', 'Serial port to use'],] class Echo(LineReceiver): def processData(self, data): print(data) def lineReceived(self, line): try: data = line.rstrip() #logging.debug(data) self.processData(data) #print(line.rstrip()) #pass except ValueError: logging.error('Unable to parse data %s' % line) return def SerialInit(): o = THOptions() try: o.parseOptions() except usage.UsageError, errortext: logging.error('%s %s' % (sys.argv[0], errortext)) logging.info('Try %s --help for usage details' % sys.argv[0]) raise SystemExit, 1 baudrate = o.opts['baudrate'] #int('115200') port = o.opts['port'] logging.debug('About to open port %s' % port) s = SerialPort(Echo(), port, reactor, baudrate=baudrate) reactor.run() thread.start_new_thread(SerialInit()) if __name__ == '__main__': print("-----") s.write('123456789') #s.write("\n")

5 7

[Twisted-Python] Defer until a server is ready?
by Russell E. Owen 21 Oct '13

21 Oct '13

For unit testing and other reasons I would like to launch a server in a subprocess, then wait until it is available before running some code that uses it. Is there a standard paradigm for this, e.g. poll until a connection is accepted, or have the server emit some kind of "ready" signal (such as a message to stdout or a UDP packet)? Any advice would be appreciated. A web search didn't turn up anything. In some cases we have used one reactor to run both the server or servers and the client, in which case waiting is trivial. But we have cases where we would much rather run the server in a separate process. -- Russell

2 1

[Twisted-Python] Weekly Bug Summary
by trac＠twistedmatrix.com 20 Oct '13

20 Oct '13

1 0

[Twisted-Python] ANN: Crochet 1.0.0 - Use Twisted Anywhere!
by Itamar Turner-Trauring 19 Oct '13

19 Oct '13

Crochet is an MIT-licensed library that makes it easier for blocking or threaded applications like Flask or Django to use the Twisted networking framework. Crochet provides the following features: * Runs Twisted's reactor in a thread it manages. * The reactor shuts down automatically when the process' main thread finishes. * Hooks up Twisted's log system to the Python standard library logging framework. Unlike Twisted's built-in logging bridge, this includes support for blocking Handler instances. * A blocking API to eventual results (i.e. Deferred instances). This last feature can be used separately, so Crochet is also useful for normal Twisted applications that use threads. You can download Crochet at: http://pypi.python.org/pypi/crochet Documentation can be found at: http://crochet.readthedocs.org Bugs and feature requests should be filed at the project page: https://github.com/itamarst/crochet *What's New in 1.0.0* ** Documentation: * Added section on use cases and alternatives. Thanks to Tobias Oberstein for the suggestion. Bug fixes: * Twisted does not have to be pre-installed to run setup.py, thanks to Paul Weaver for bug report and Chris Scutcher for patch. * Importing Crochet does not have side-effects (installing reactor event) any more. * Blocking calls are interrupted earlier in the shutdown process, to reduce scope for deadlocks. Thanks to rmorehead for bug report. * *

1 0

[Twisted-Python] Setting source address of outgoing datagrams
by Maciej Wasilak 18 Oct '13

18 Oct '13

Hello, I've got a problem when using Twisted UDP-based app with IPv6. I know UDP IPv6 is not yet officially supported, I am using the unoffcial hack found somewhere on the net. ----------------UNOFFICIAL HACK---------------------- In twisted/internet/udp.py: (...) def __init__(self, port, proto, interface='', maxPacketSize=8192, reactor=None): """ Initialize with a numeric port to listen on. """ base.BasePort.__init__(self, reactor) self.port = port self.protocol = proto self.maxPacketSize = maxPacketSize self.interface = interface self.setLogStr() self._connectedAddr = None + if interface and abstract.isIPv6Address(interface): ### + self.addressFamily = socket.AF_INET6 ### (...) def doRead(self): """ Called when my socket is ready for reading. """ read = 0 while read < self.maxThroughput: try: data, addr = self.socket.recvfrom(self.maxPacketSize) except socket.error as se: no = se.args[0] if no in _sockErrReadIgnore: return if no in _sockErrReadRefuse: if self._connectedAddr: self.protocol.connectionRefused() return raise else: read += len(data) try: + addr = (addr[0], addr[1]) ### self.protocol.datagramReceived(data, addr) except: log.err() --------------END OF UNOFFICIAL HACK------------------------ I deploy my protocol like that: reactor.listenUDP(5683, myapp.MyProtocol(), interface="::") reactor.run() The problem is: 1. My server receives a request from remote client (with destination address being valid global IPv6 address) 2. The server sends a response, but uses different source address (which is also valid and assigned to local wlan0 interface). 3. Remote host drops the response (destination address of the request and source address of the response do not match). It is common for IPv6 interfaces to have multiple global IPv6 addresses - because of privacy reasons (both on Windows and Linux). My question is - can I read manually the destination address from the incoming request datagram, and set it as a source address manually in the outgoing response datagram? Best Regards Maciej Wasilak

3 5

[Twisted-Python] pb.Cacheable with response to remote requests
by Daniel Sank 17 Oct '13

17 Oct '13

Is there a good way to make something like a pb.Cacheable on which the observers can call remote methods? -- Daniel Sank Department of Physics Broida Hall University of California Santa Barbara, CA 93117 (805)893-3899

1 0