Mailman 3 February 2010 - Twisted

[Twisted-Python] twisted.protocol.sip
by Lorenzo Mainardi Feb. 23, 2010

Feb. 23, 2010

Hello, I'm trying to extend the class twisted.protocols.sip.MessageParser for create my parser. I had read the documentation and I found this: "Shouldn't be connected to actual transport.". What does it means? I can't use that binding it to a socket? I found also that many other object in twisted.protocols.sip are not complete and/or not working (some of that return everytime NotImplementedYet). Is it correct or I'm doing some mistakes? -- LORENZO MAINARDI Email: lormayna(a)gmail.com Linux … [View More]

2 1

Re: [Twisted-Python] Try/catching yielded Exceptions and getting proper tracebacks
by Terry Jones Feb. 23, 2010

Feb. 23, 2010

Hi Paul >>>>> "Paul" == Paul Goins <general(a)vultaire.net> writes: You're only printing the exception, not a full traceback, so you don't see much. I tend to write what you're doing as follows: from twisted.python import log @defer.inlineCallbacks def xmlrpc_dosomething(self): d = self._do_something_else() d.addErrback(log.err) result = yield d defer.returnValue(result) If you try that you'll see a full traceback. The above … [View More]

2 1

[Twisted-Python] Looking for companies doing Twisted
by Nicolas Dietrich Feb. 22, 2010

Feb. 22, 2010

Hi there, I'm not sure if this is the right place to ask, so sorry for the spam in advance. Anyway: I'm looking for companies which might be interested in developing a booking system for an emerging Germany-based long-distance railway undertaking during this year. This will be a distributed system, so Twisted might be a good solution for this. Please approach me directly for any details. Thankful for all hints, Nicolas

1 0

[Twisted-Python] Try/catching yielded Exceptions and getting proper tracebacks
by Paul Goins Feb. 22, 2010

Feb. 22, 2010

I'm frequently using the inlineCallbacks idiom for writing code, but am often finding myself frustrated with rather counterintuitive tracebacks. Let's say I'm writing an XML-RPC handler, and even if an exception occurs, I want it to return something to the user and dump the exception to the log file. However, if an exception occurs when I'm using the inlineCallbacks decorator, I sometimes get lacking tracebacks. Maybe I'll have the line number of the function that yielded the exception, but … [View More]

1 0

[Twisted-Python] Many connections and TIME_WAIT
by Donal McMullan Feb. 22, 2010

Feb. 22, 2010

I've been prototyping a client that connects to thousands of servers and calls some method. It's not real important to me at this stage whether that's via xmlrpc, perspective broker, or something else. What seems to happen on the client machine is that each network connection that gets opened and then closed goes into a TIME_WAIT state, and eventually there are so many connections in that state that it's impossible to create any more. I'm keeping an eye on the output of netstat -an | wc -l … [View More]

2 2

[Twisted-Python] Nevow question
by Sury Soni Feb. 21, 2010

Feb. 21, 2010

Hi, How do we disable traceback on Nevow production site? Right now, by default, any nevow site will print full length of traceback on web browser if there is some exception within the application. I have gone through the ticket http://twistedmatrix.com/trac/ticket/135 but could not find quick place to disable/enable the same in my nevow application. Sury Prakash Soni

2 2

[Twisted-Python] fast high load protocol
by Vlad Shevchenko Feb. 21, 2010

Feb. 21, 2010

Hi, I have developed two kind of servers: api - http server memory_db - dict-based server with some logic of putting data to dict and getting it back. Api communicate with memory_db via txJSON-RPC but response time is huge under high load. I also try implementation with replacement txJSON-RPC -> Perspective Broker with no success. PB implementation has 10+ times faster responses when returning data is small (dict with few keys and values like list of 1-5 strings). However, PM much … [View More]

5 8

[Twisted-Python] Need help for structured application development
by Einar S. Idsø Feb. 21, 2010

Feb. 21, 2010

Hi, I am planning to develop an application that will query a number of services at given intervals, and Twisted seems like a very nice platform to do so. However, because of little previous experience with Twisted, especially with building proper applications, using interfaces and using plugins, I could really need some pointers to get me started. In the following I will try to explain in broad terms what the application will do, and humbly ask for input on how I may structure it using … [View More]factories, protocols etc.: 1. A number of more-or-less similar server applications (5-10) have a number of concurrent instances/servers (1-30) each. 2. I need to acquire the status of each instance at given intervals. These will typically range from every 20 seconds to every 3 minutes, and depend on the which service the instance is based on. 3. A list of the server instances shall be acquired via an XML-query every few minutes to pick up on new or removed instances. 4. An executable already exists which can be used to status-poll almost all services using different parameters. I will be calling it from my application to do the polling instead of recreating it in Python. 5. I need to be able to create my own polling-protocol for at least one of the service types which is incompatible with the executable. 6. Some services can not only be polled at intervals, but additionally stream information that I am interested in listening to. 7. At a later point in time, I wish to be able to communicate with IRC, MSN and other services/protocols using the queried and streamed data. My initial general thought is the following: 1. Design a class for each server application, where all classes implement the same interface. The class defines how to deal with server instances of its corresponding server application type. Classes are created as Twisted-plugins. 2. For each new server instance that is discovered, create an object which contains a timer and polls the server at specific intervals, writing the result to a globally accessible data structure. 3. For server instances that stream data continuously in addition to being polled at regular intervals, create a separate object which listens to the server. I believe I should somehow have a global data structure which contains references to all the server instance objects, regardless of type, and allows them to store data. This is sort of like the factory does for protocols, except in my case I don't think I can use a regular factory since I need both clients (to poll at intervals) and servers (to listen to streaming data). I suppose I could use a ClientFactory for the pollers and a Factory for the stream-listeners, and let them have a common data-store? But is it possible to use a single ClientFactory to spawn all poller-objects regardless of server type? Or do I need one ClientFactory for each server type? Do I perhaps need one ClientFactory per server instance? Or should I just ignore factories all together since it is primarily the timer and asyncronicity of Twisted I need? I am quite certain there is a correct twistified way of handling this sort of problem. I am just not sure what it is, and would sincerely welcome all feedback on how this should be done. Feel free to point to examples of applications that are similar to what I sketched above, and tell me to study the code :) Cheers, Einar [View Less]

4 7

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Feb. 21, 2010

Feb. 21, 2010

1 0

[Twisted-Python] Streaming File Transfer Protocol?
by Darren Govoni Feb. 19, 2010

Feb. 19, 2010

Hi, Is there an existing protocol that can provide the following? - Accept stream binary data FROM a client (e.g. very large file transfer) - Receive data IN ORDER (i.e. stream. not out of order random packets) I want to stream FROM a client to the protocol server and have the server process the stream bytes incrementally so it doesn't have to store or write the entire data stream (too large). I looked at FileTransferServer and Client, but I'm not sure it provides what I need. Any tips appreciated! Darren

10 27

[Twisted-Python] Running php script on twisted server
by Tux Attack Feb. 18, 2010

Feb. 18, 2010

I am trying to run a php script on a webserver written in twisted.For that i am using twcgi module. Here is my code. This is code for webServer.py -> from twisted.internet import reactor from twisted.web import static,script,twcgi,server from twisted.python.log import startLogging import sys class PHPScript(twcgi.FilteredScript): filter='/media/DATA/Documents/project-docs/twisted/twisted-programs/testPHP/twisted-php.py' root=static.File('html_files') root.indexNames=['index.php'] root.… [View More]

1 0

[Twisted-Python] Authenticating with md5 hashed passwords
by Ramiro Alba Queipo Feb. 17, 2010

Feb. 17, 2010

Hello everybody, I am trying to build a a client/server application using Perspective Broker and wanting to authenticate against a PostgreSQL database. Everything works fine if I user plain text passwords, but when trying to hash then using md5 using from hashlib import md5 md5Password = md5(password).hexdigest() then it does not authenticate (I use credentials.checkMD5Password(password) at the checker class) Then after reading twisted/spread/pb.py I saw that everything is done in the … [View More]

10 22

[Twisted-Python] Help Twisted at PyCon (or anywhere else)
by Itamar Turner-Trauring Feb. 17, 2010

Feb. 17, 2010

Hello Twisted users, Does Twisted help you with your job, hobby or homework? Here's how you can help Twisted in return, especially if you're coming to PyCon: * Talk to us! We want to hear what features you want, what problems you are having. But especially, we want to hear about the cool stuff you build with Twisted, because that makes us happy (and even happier to work on Twisted and help you.) * Develop Twisted! Whether it's writing documentation or fixing bugs, we always appreciate help … [View More]

1 0

Re: [Twisted-Python] web page on doc...
by Kevin Horn Feb. 17, 2010

Feb. 17, 2010

On Tue, Feb 16, 2010 at 2:18 PM, K. Richard Pixley <rich(a)noir.com> wrote: > Kevin Horn wrote: > > xhtml is processed into html by "lore" not "trial". > > Doh. Thanks. I *knew* that. I haven't gotten either to work yet, but > neither is really on my radar right now either. > > Well, I haven't gotten Lore to work (on Windows at least) in a looooooong time. The only way I can get trial to work on Windows is by using the "Twisted Command Prompt", or some hairy … [View More]

2 1

[Twisted-Python] web page on doc...
by K. Richard Pixley Feb. 16, 2010

Feb. 16, 2010

I've been going around and around on documentation with/for twisted for a few days now. For example, I read things like "the documentation is written in epytext" and "documentation is processed by trial" and conclude that trial processes epytext. I think I'm beginning to understand what's really going on now but two things would have helped me. 1) a brief overview of the documentation. (proposal below). 2) if references to "the documentation" could be clarified to refer to "howto … [View More]

2 1

[Twisted-Python] reactor.run hangs
by giskard Feb. 15, 2010

Feb. 15, 2010

Hi all, i think i'm not doing things in the right way: http://paste.debian.net/59855/ this code implement a simple jabber client, if you remove the class defnition and run it (removing also the run def) it works! but if i 'class' it and run via: http://paste.debian.net/59856/ reactor.run() hangs. where i'm wrong? thank you in advance, -- ciao, giskard

3 3

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Feb. 14, 2010

Feb. 14, 2010

1 0

[Twisted-Python] newbie confusion - puzzling reactor response
by K. Richard Pixley Feb. 13, 2010

Feb. 13, 2010

I'm confused be the response I get to the attached program. In a nutshell, I'm building a reader, attaching it with addReader, later removing it with removeReader. And I'm getting this: time python test_reactor.py Traceback (most recent call last): Failure: twisted.internet.error.ConnectionFdescWentAway: Uh: Filedescriptor went away. Which seems to be telling me that I don't know as much yet as I'd hoped. Why would the reactor care about a closed file descriptor that isn't … [View More]

4 11

[Twisted-Python] new epoll error after upgrading to 9.0.0
by Alec Matusis Feb. 12, 2010

Feb. 12, 2010

I upgraded to 9.0.0 and I am now seeing a new error, not present in 8.2.0 or earlier: 2010-02-10 17:38:33-0800 [TagProtocol,9794986,68.126.204.104] Unhandled Error Traceback (most recent call last): File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/application/app.py", line 348, in runReactorWithLogging reactor.run() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-… [View More]py2 .6-linux-x86_64.egg/twisted/internet/base.py", line 1166, in run self.mainLoop() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/base.py", line 1178, in mainLoop self.doIteration(t) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 194, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 223, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/posixbase.py", line 188, in _disconnectSelectable selectable.readConnectionLost(f) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/tcp.py", line 508, in readConnectionLost self.connectionLost(reason) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/tcp.py", line 513, in connectionLost abstract.FileDescriptor.connectionLost(self, reason) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/abstract.py", line 67, in connectionLost self.stopWriting() File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/abstract.py", line 267, in stopWriting self.reactor.removeWriter(self) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 145, in removeWriter self._remove(writer, self._writes, self._reads, self._selectables, _epoll.OUT, _epoll.IN) File "/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2 .6-linux-x86_64.egg/twisted/internet/epollreactor.py", line 131, in _remove self._poller._control(cmd, fd, flags) File "_epoll.pyx", line 125, in _epoll.epoll._control exceptions.IOError: [Errno 2] No such file or directory The error is highy intemittent and occurs only under high connection client rate. Any idea of what this could be? [View Less]

2 1

[Twisted-Python] How can I send unique configuration data for multiple connections through a factory?
by Mark Bailey Feb. 10, 2010

Feb. 10, 2010

Good day: I've been having fun with Twisted. I have my application running fine, with multiple server and client connections using Telnet. :-) However, users always want something. I need to send some unique configuration information to each connection. The connections are made using connectTCP. My first attempt assumed that the connection was made when the call to connectTCP was executed. It didn't work! :-( ------------------------------- from twisted.internet.protocol import … [View More]ClientFactory from twisted.internet import reactor from twisted.conch.telnet import StatefulTelnetProtocol class testClient(StatefulTelnetProtocol): def connectionMade(self): self.title = self.factory.connectString print "Client Connected: " + self.title self.setRawMode() self.factory.connections.append(self) def connectionLost(self, reason): if self in self.factory.connections: self.factory.connections.remove(self) def rawDataReceived(self,data): print data + "\n" class ClusterClientFactory(ClientFactory): protocol = testClient def __init__(self): self.connections = [] self.connectString = '' def startFactory(self): print "startFactory: " + self.connectString def startedConnecting(self, connector): print "Started connecting: " + str(connector) print self.connectString def buildProtocol(self, addr): print "bulldProtocol: " + str(addr) print "buildProtocol: " + self.connectString p = self.protocol() p.factory = self return p if __name__ == '__main__': def startUp(): factory = ClusterClientFactory() factory.maxDelay = 120 # two minutes factory.connectString = "FirstString..." reactor.connectTCP("localhost", 7300, factory) factory.connectString = "SecondString" reactor.connectTCP("localhost", 7300, factory) reactor.callWhenRunning(startUp) reactor.run() ---------------------------------- The results are that buildProtocol gets the second string both times. startedConnecting gets the correct string, but all I have there is a connection object. I need the correct data in buildProtocol to do it this way. ------------------------------------ C:\Users\Mark\src\play>python testclient.py startFactory: FirstString... Started connecting: <twisted.internet.tcp.Connector instance at 0x01DAB620> FirstString... Started connecting: <twisted.internet.tcp.Connector instance at 0x01E0EFD0> SecondString bulldProtocol: IPv4Address(TCP, '127.0.0.1', 7300) buildProtocol: SecondString Client Connected: SecondString Welcome to SimpleServer5 bulldProtocol: IPv4Address(TCP, '127.0.0.1', 7300) buildProtocol: SecondString Client Connected: SecondString Welcome to SimpleServer5 -------------------------- My "best" idea is to pass the strings to the factory in a dictionary indexed by the IP address and the port. Then, buildProtocol() can use that to recover the string and I can use reactor.resolve() to translate the host name to the IP address. The real application won't have multiple connections to the same host and port like this example, so this would work. There MUST be a better way. :-) My "simpleserver" is below if you want to run this. THANKS for all of your help. I have the basic application running now (including a Tkinter GUI :-) ) and Twisted has saved me hundreds or even thousands of lines of code... Mark Bailey ------------------------------ from twisted.conch.telnet import StatefulTelnetProtocol from twisted.internet import reactor, protocol from twisted.protocols.basic import LineReceiver class TelnetEcho(StatefulTelnetProtocol): def connectionMade(self): self.factory.connection.append(self) self.sendLine("Welcome to SimpleServer5\r\n") def lineReceived(self, data): data = data.rstrip('\n\r') if data.upper() == 'BYE': self.sendLine("Goodbye...\r") self.transport.loseConnection() else: self.sendLine("Unrecognized command: %r\r" % (data,)) def connectionLost(self, reason): self.factory.connection.remove(self) class TelnetEchoFactory(protocol.Factory): protocol = TelnetEcho def __init__(self): self.connection = [] def createTelnetServer(port=7300): telnetinstance = TelnetEchoFactory() # needs to be a list reactor.listenTCP(port,telnetinstance) if __name__ == "__main__": reactor.callWhenRunning(createTelnetServer) reactor.run() [View Less]

2 2

[Twisted-Python] Light-est-weight http authentication
by Brad Milne Feb. 10, 2010

Feb. 10, 2010

I have a backend Twisted application with a front end in Django. Some of their communication is via web services. Currently they are deployed on the same box but may be separated in future. I want to enforce some basic authentication on the web services, so some lightweight http header authentication looks best. I've found the twisted.web.guard api, but it seems to be more heavyweight than I require (e.g. running up a Portal). Given the breadth of the Twisted apis, I'm confident I wouldn't have … [View More]

3 7

[Twisted-Python] Error handling for modules which use twisted.enterprise.adbapi
by Vishal Shetye Feb. 10, 2010

Feb. 10, 2010

Hi, I have an application that uses twisted.enterprise.adbapi for accessing sqlite database. Recently I had to replace sqlite3 shipped with python 2.5 with latest version of pysqlite2. This was for taking advantage of newer fixes in pysqlite2. I thought it would be worthwhile to support both. However, functions called using runInteraction have a code block as given below. try: txn.execute(“update query”) except sqlite3.IntegrityError: handleDuplicateValues() or an errback on the … [View More]

1 0

[Twisted-Python] PyCon 2010 Twisted Sprint
by exarkun＠twistedmatrix.com Feb. 8, 2010

Feb. 8, 2010

Hi all, If there are enough interested people, there will be a Twisted sprint at PyCon this year. If you'll be at PyCon and are interested in sprinting on some Twisted topic, give a shout. Also, let us know what specific topics you're interested in so we can let other people know what to expect in the general announcement. Jean-Paul

9 10

[Twisted-Python] Handling exceptions in twisted.internet.udp?
by Attila Nagy Feb. 8, 2010

Feb. 8, 2010

Hello, I have a code, which opens persistent (connected, see: http://twistedmatrix.com/documents/current/core/howto/udp.html#auto2) UDP connections, and sends and receives data on them. The problem is that occasionally I receive ICMP 13 (administratively prohibited) errors from the destinations, which triggers and exception and closes the UDP socket. I would like to keep that open, no matter what happens. On FreeBSD the exception comes during writing into the socket with transport.write,… [View More] so this is an easy one. But on Linux it's a little bit harder, because there I get exception not during writing, but reading, which is handled here: http://twistedmatrix.com/trac/browser/tags/releases/twisted-9.0.0/twisted/i… The exception looks like this: 2010-02-08 12:09:15+0100 [-] reactor class: twisted.internet.selectreactor.SelectReactor. 2010-02-08 12:10:22+0100 [__builtin__.connectedUDP (UDP)] Unhandled Error Traceback (most recent call last): File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/context.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/python/context.py", line 37, in callWithContext return func(*args,**kw) --- <exception caught here> --- File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/internet/selectreactor.py", line 146, in _doReadOrWrite why = getattr(selectable, method)() File "/usr/lib64/python2.6/site-packages/Twisted-9.0.0_r27775-py2.6-linux-x86_64.egg/twisted/internet/udp.py", line 109, in doRead data, addr = self.socket.recvfrom(self.maxPacketSize) socket.error: [Errno 113] No route to host I wonder what is the preferred way of achieving that the socket will be write()-able and kept open regardless what ICMP or other sending or receiving error occurs? Thanks, [View Less]

2 2

[Twisted-Python] domish.Element.__getattr__() or xpath.XPathQuery
by Gabriel Rossetti Feb. 8, 2010

Feb. 8, 2010

Hello everyone, I was wondering what you would recommend, accessing nodes via domish.Element.__getattr__() (e.g. el.node1.node1_1, where el is /msg) or via xpath.XPathQuery() (e.g. xpath.XPathQuery("/msg/node1/node1_1").queryForNodes(el)[0])? I was wondering which would be faster, the __getattr__() is O(n) (in my example there are 2 calls) but what about xpath.XPathQuery()? Thank you, Gabriel

3 4