Mailman 3 February 2010 - Twisted

[Twisted-Python] reactor.run hangs
by giskard 15 Feb '10

15 Feb '10

Hi all, i think i'm not doing things in the right way: http://paste.debian.net/59855/ this code implement a simple jabber client, if you remove the class defnition and run it (removing also the run def) it works! but if i 'class' it and run via: http://paste.debian.net/59856/ reactor.run() hangs. where i'm wrong? thank you in advance, -- ciao, giskard

3 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 13 Feb '10

13 Feb '10

1 0

[Twisted-Python] newbie confusion - puzzling reactor response
by K. Richard Pixley 12 Feb '10

12 Feb '10

I'm confused be the response I get to the attached program. In a nutshell, I'm building a reader, attaching it with addReader, later removing it with removeReader. And I'm getting this: time python test_reactor.py Traceback (most recent call last): Failure: twisted.internet.error.ConnectionFdescWentAway: Uh: Filedescriptor went away. Which seems to be telling me that I don't know as much yet as I'd hoped. Why would the reactor care about a closed file descriptor that isn't even in it's interest set? --rich #!/usr/bin/env python # -*- coding: utf-8 -*- import os from zope.interface import implements from twisted.internet import reactor from twisted.internet.interfaces import IReadDescriptor class inputFile(object): implements(IReadDescriptor) def __init__(self, filename): self.filename = filename self.filedes = os.open(filename, os.O_RDONLY | os.O_NONBLOCK) reactor.addReader(self) def fileno(self): return self.filedes def connectionLost(self, reason): raise reason def logPrefix(self): return 'inputFile' def doRead(self): reactor.removeReader(self) os.close(self.filedes) self.filedes = -1 reactor.stop() if __name__ == '__main__': r = inputFile('/etc/group') reactor.addReader(r) reactor.run()

4 11

[Twisted-Python] new epoll error after upgrading to 9.0.0
by Alec Matusis 11 Feb '10

11 Feb '10

I upgraded to 9.0.0 and I am now seeing a new error, not present in 8.2.0 or earlier: 2010-02-10 17:38:33-0800 [TagProtocol,9794986,68.126.204.104] Unhandled Error Traceback (most recent call last): File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/application/app.py", line 348, in runReactorWithLogging reactor.run() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/base.py", line 1166, in run self.mainLoop() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/base.py", line 1178, in mainLoop self.doIteration(t) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 194, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 223, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/posixbase.py", line 188, in _disconnectSelectable selectable.readConnectionLost(f) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/tcp.py", line 508, in readConnectionLost self.connectionLost(reason) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/tcp.py", line 513, in connectionLost abstract.FileDescriptor.connectionLost(self, reason) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/abstract.py", line 67, in connectionLost self.stopWriting() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/abstract.py", line 267, in stopWriting self.reactor.removeWriter(self) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 145, in removeWriter self._remove(writer, self._writes, self._reads, self._selectables, _epoll.OUT, _epoll.IN) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 131, in _remove self._poller._control(cmd, fd, flags) File "_epoll.pyx", line 125, in _epoll.epoll._control exceptions.IOError: [Errno 2] No such file or directory The error is highy intemittent and occurs only under high connection client rate. Any idea of what this could be?

2 1

[Twisted-Python] How can I send unique configuration data for multiple connections through a factory?
by Mark Bailey 10 Feb '10

10 Feb '10

Good day: I've been having fun with Twisted. I have my application running fine, with multiple server and client connections using Telnet. :-) However, users always want something. I need to send some unique configuration information to each connection. The connections are made using connectTCP. My first attempt assumed that the connection was made when the call to connectTCP was executed. It didn't work! :-( ------------------------------- from twisted.internet.protocol import ClientFactory from twisted.internet import reactor from twisted.conch.telnet import StatefulTelnetProtocol class testClient(StatefulTelnetProtocol): def connectionMade(self): self.title = self.factory.connectString print "Client Connected: " + self.title self.setRawMode() self.factory.connections.append(self) def connectionLost(self, reason): if self in self.factory.connections: self.factory.connections.remove(self) def rawDataReceived(self,data): print data + "\n" class ClusterClientFactory(ClientFactory): protocol = testClient def __init__(self): self.connections = [] self.connectString = '' def startFactory(self): print "startFactory: " + self.connectString def startedConnecting(self, connector): print "Started connecting: " + str(connector) print self.connectString def buildProtocol(self, addr): print "bulldProtocol: " + str(addr) print "buildProtocol: " + self.connectString p = self.protocol() p.factory = self return p if __name__ == '__main__': def startUp(): factory = ClusterClientFactory() factory.maxDelay = 120 # two minutes factory.connectString = "FirstString..." reactor.connectTCP("localhost", 7300, factory) factory.connectString = "SecondString" reactor.connectTCP("localhost", 7300, factory) reactor.callWhenRunning(startUp) reactor.run() ---------------------------------- The results are that buildProtocol gets the second string both times. startedConnecting gets the correct string, but all I have there is a connection object. I need the correct data in buildProtocol to do it this way. ------------------------------------ C:\Users\Mark\src\play>python testclient.py startFactory: FirstString... Started connecting: <twisted.internet.tcp.Connector instance at 0x01DAB620> FirstString... Started connecting: <twisted.internet.tcp.Connector instance at 0x01E0EFD0> SecondString bulldProtocol: IPv4Address(TCP, '127.0.0.1', 7300) buildProtocol: SecondString Client Connected: SecondString Welcome to SimpleServer5 bulldProtocol: IPv4Address(TCP, '127.0.0.1', 7300) buildProtocol: SecondString Client Connected: SecondString Welcome to SimpleServer5 -------------------------- My "best" idea is to pass the strings to the factory in a dictionary indexed by the IP address and the port. Then, buildProtocol() can use that to recover the string and I can use reactor.resolve() to translate the host name to the IP address. The real application won't have multiple connections to the same host and port like this example, so this would work. There MUST be a better way. :-) My "simpleserver" is below if you want to run this. THANKS for all of your help. I have the basic application running now (including a Tkinter GUI :-) ) and Twisted has saved me hundreds or even thousands of lines of code... Mark Bailey ------------------------------ from twisted.conch.telnet import StatefulTelnetProtocol from twisted.internet import reactor, protocol from twisted.protocols.basic import LineReceiver class TelnetEcho(StatefulTelnetProtocol): def connectionMade(self): self.factory.connection.append(self) self.sendLine("Welcome to SimpleServer5\r\n") def lineReceived(self, data): data = data.rstrip('\n\r') if data.upper() == 'BYE': self.sendLine("Goodbye...\r") self.transport.loseConnection() else: self.sendLine("Unrecognized command: %r\r" % (data,)) def connectionLost(self, reason): self.factory.connection.remove(self) class TelnetEchoFactory(protocol.Factory): protocol = TelnetEcho def __init__(self): self.connection = [] def createTelnetServer(port=7300): telnetinstance = TelnetEchoFactory() # needs to be a list reactor.listenTCP(port,telnetinstance) if __name__ == "__main__": reactor.callWhenRunning(createTelnetServer) reactor.run()

2 2

[Twisted-Python] Light-est-weight http authentication
by Brad Milne 10 Feb '10

10 Feb '10

I have a backend Twisted application with a front end in Django. Some of their communication is via web services. Currently they are deployed on the same box but may be separated in future. I want to enforce some basic authentication on the web services, so some lightweight http header authentication looks best. I've found the twisted.web.guard api, but it seems to be more heavyweight than I require (e.g. running up a Portal). Given the breadth of the Twisted apis, I'm confident I wouldn't have to hand-craft something to check the auth header myself. What do you guys suggest? Thanks Brad

3 7

[Twisted-Python] Error handling for modules which use twisted.enterprise.adbapi
by Vishal Shetye 10 Feb '10

10 Feb '10

Hi, I have an application that uses twisted.enterprise.adbapi for accessing sqlite database. Recently I had to replace sqlite3 shipped with python 2.5 with latest version of pysqlite2. This was for taking advantage of newer fixes in pysqlite2. I thought it would be worthwhile to support both. However, functions called using runInteraction have a code block as given below. try: txn.execute(“update query”) except sqlite3.IntegrityError: handleDuplicateValues() or an errback on the function runQuery/Interaction/Operation. They all used twisted.python.failure.check/trap on some exception of the form sqlite3.some_dbapi_exception. Now, all such places need to import exceptions from pysqlite2 instead of sqlite3. However, to handle multiple dbapi modules one of which will be configured for use by the user, I thought, I will have a global name having reference of the current dbapi module being used. This will be used by errbacks and except blocks. Is there a better way of handling exceptions in such cases? I came across http://svn.sqlalchemy.org/sqlalchemy/trunk/lib/sqlalchemy/exc.py. I think sqlalchemy wraps dbapi erros in DBAPIError exception. I imagine doing something similar in twisted as, try: txn.execute(“update query”) except adbapi.IntegrityError: handleDuplicateValues() I also came across http://mail.python.org/pipermail/db-sig/2003-September/003686.html which suggests exceptions raised by different modules for same type of error may not be consistent, atleast with respect to the information provided. Any suggestions on this would be very helpful. Just to note http://twistedsphinx.funsize.net/projects/core/howto/rdbms.html does not have error handling related info. -- vishal shetye

1 0

[Twisted-Python] PyCon 2010 Twisted Sprint
by exarkun＠twistedmatrix.com 08 Feb '10

08 Feb '10

Hi all, If there are enough interested people, there will be a Twisted sprint at PyCon this year. If you'll be at PyCon and are interested in sprinting on some Twisted topic, give a shout. Also, let us know what specific topics you're interested in so we can let other people know what to expect in the general announcement. Jean-Paul

9 10

[Twisted-Python] Handling exceptions in twisted.internet.udp?
by Attila Nagy 08 Feb '10

08 Feb '10

Hello, I have a code, which opens persistent (connected, see: http://twistedmatrix.com/documents/current/core/howto/udp.html#auto2) UDP connections, and sends and receives data on them. The problem is that occasionally I receive ICMP 13 (administratively prohibited) errors from the destinations, which triggers and exception and closes the UDP socket. I would like to keep that open, no matter what happens. On FreeBSD the exception comes during writing into the socket with transport.write, so this is an easy one. But on Linux it's a little bit harder, because there I get exception not during writing, but reading, which is handled here: http://twistedmatrix.com/trac/browser/tags/releases/twisted-9.0.0/twisted/i… The exception looks like this: 2010-02-08 12:09:15+0100 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2010-02-08 12:10:22+0100 [__builtin__.connectedUDP (UDP)] Unhandled Error Traceback (most recent call last): File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) --- <exception caught here> --- File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite why = getattr(selectable, method)() File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/internet/udp.py", line 109, in doRead data, addr = self.socket.recvfrom(self.maxPacketSize) socket.error: [Errno 113] No route to host I wonder what is the preferred way of achieving that the socket will be write()-able and kept open regardless what ICMP or other sending or receiving error occurs? Thanks,

2 2

[Twisted-Python] domish.Element.__getattr__() or xpath.XPathQuery
by Gabriel Rossetti 08 Feb '10

08 Feb '10

Hello everyone, I was wondering what you would recommend, accessing nodes via domish.Element.__getattr__() (e.g. el.node1.node1_1, where el is /msg) or via xpath.XPathQuery() (e.g. xpath.XPathQuery("/msg/node1/node1_1").queryForNodes(el)[0])? I was wondering which would be faster, the __getattr__() is O(n) (in my example there are 2 calls) but what about xpath.XPathQuery()? Thank you, Gabriel

3 4