Mailman 3 April 2008 - Twisted

[Twisted-Python] about using twisted to write a RT server.
by Chris 23 Apr '08

23 Apr '08

Hi everyone, I want to write a server,it works just like this: 1.client will connect to the server,maybe with some auth information. 2.the server verify the client,if it's ok,accept the connection. 3.the client starts to sent some binary data to server buffer by buffer ,the server will use it to calculate in real time. 4.when all the data is sent,the client will tell the server no data any more. 5.the server gets a result from the data,and sends the result to the client(the client is ASCII code) 6.server shuts down the connection. first,I want to know which protocol I should use?the LineReceiver is ok for this? second,I know the LineReceiver protocol can switch between line mode and row mode,I think I will use line mode to send the control command,and use raw mode to sent the data,but how the client can inform the server which mode should use?because sometimes the data which is received in raw mode is exactly a legal command,but it is just only a part of the data. third,I don't know very much about how to send the binary data,I heard I should use struct modul,is it right? any hint is helpful,please just give me a direction,I hope my question will not take too much time to answer,thanks a lot. Regards Chris

1 0

[Twisted-Python] Location of remote errors with PB
by Simon Pickles 23 Apr '08

23 Apr '08

Hello, my app uses twisted.pb to communicate across its parts. Is there any way a caller can find out the whole traceback rather than just the description( eg "*global name 'true' is not defined*"?) This is so I can have help debugging. Thanks Si -- Linux Counter: User# 424693

5 7

[Twisted-Python] Big/Little-endian problem
by Xian Chen 22 Apr '08

22 Apr '08

Hi All, I am newbie to Twisted. In Twisted, do the user need to take care of the big/little-endian issues? For example, I receive a buffer via UDP which actually represents a unsinged int value. So, how can i convert this buffer[0:1] to unsigned int ? Thanks, Xian

2 2

Re: [Twisted-Python] Why is stopProducing() called?
by Andrew Francis 22 Apr '08

22 Apr '08

Hi Gabriel: >Is there a way to log the execution of twisted? I'm >getting a connection loss and I don't know where it >comes from and why. I'm not explicitly calling >looseConnection. I think the easiest way to do logging is from twisted.python import log ... log.startLogging(a file descriptor - i.e., sys.stdout) Twisted will start emitting log messages. You can use log.msg() for additional messages. I also found using a debugger is a slow but good way to trace through what Twisted is doing. Cheers, Andrew ____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

2 1

[Twisted-Python] what is the best way to serve static files?
by inhahe 22 Apr '08

22 Apr '08

What is the best (fastest) way to serve static files with Twisted? Or would using Apache or lighttpd for static files significantly speed it up (i don't know how to do this, btw)? Thx.

3 2

[Twisted-Python] question about twisted web2
by inhahe 22 Apr '08

22 Apr '08

i may not understand all this very well, but, if i remember correctly, http 1.1 supports multiple requests over the same connection to speed things up.. but http://twistedmatrix.com/projects/web/documentation/howto/web-overview.html… me to believe it closes the connection after each request.. but twisted web2, otoh, is supposed to have full 1.1 support.. does this mean it won't close the connection after each request? if so, i'd like to use it for that reason, but then, according to http://twistedmatrix.com/trac/wiki/TwistedWeb2 web2 isn't very fast. i don't know if their benchmarks take under consideration realistic conditions in which multiple requests can be made over a single tcp connection. so should i use web2, and if not, is there any way to do what i want using twisted.web, and if not, what python web framework can i use that does do what i want and is otherwise as fast as twisted web?

3 3

[Twisted-Python] Why is stopProducing() called?
by Gabriel Rossetti 22 Apr '08

22 Apr '08

Hello, I keep on getting stopProducing() called on my push producer (yes, I'm going to try it anyways). From what I read, that means that the consumer died, correct? What can make the consumer die? My consumer is a TCP/IP transport and all my tests are done through the loopback interface, thus the network can't be down, any ideas? Gabriel

1 1

[Twisted-Python] April 20th Sprint Report
by Jean-Paul Calderone 22 Apr '08

22 Apr '08

Hello all, The first TSF sponsored Twisted sprint is done. Here's a quick rundown of some of the things that happened. * Glyph Lefkowitz made a huge dent in the current review queue, dealing with more than half of the outstanding tickets. * After some discussion about the value of a bugfix-only release vs a normal bugfix and feature enhancement release, Christopher Armstrong got the wheels rolling on 8.1.0 (which will be of the latter type). You probably already saw the pre1 announcement. Give it a spin! * Paul Swartz continued to strive towards the goal of Twisted Conch being as soft and cuddly as a little bunny rabbit. * Ying Li worked on permission and ownership features for FilePath. * Jeff Mancuso came by for a while and talked about Twisted Conch performance (and did some benchmarks) and SFTP access control issues. * Roland Hedberg got his hands dirty in pyOpenSSL working on adding an API to get arbitrary X509v3 extensions. * Itamar Shtull-Trauring helped out all around. * I finished fixing some reactor startup and shutdown issues introduced in 8.0. We also discussed Twisted's backwards compatibility policy, guided mostly by Jonathan Lange's email of the 18th to this list, with the goal of creating an official document. The next sprint is tentatively scheduled for May 18th at the Divmod office. Jean-Paul

3 2

[Twisted-Python] "Import Error" after installing twisted matrix
by Ben 21 Apr '08

21 Apr '08

Hi, I am having problem compiling twisted sample codes. It gives me "Import Error" errors (i.e. ImportError: No module named web.client). I really appreciate any suggestions. Thank you. Below is the sample codes that I tried from the tutorial *from twisted.web.client import getPage from twisted.internet import reactor def printContents (contents): ''' This is the 'callback' function, added to the Deferred and called by it when the promised data is available ''' print "The Deferred has called printContents with the following contents:" print contents # Stop the Twisted event handling system -- this is usually handled # in higher level ways reactor.stop() # call getPage, which returns immediately with a Deferred, promising to # pass the page contents onto our callbacks when the contents are available deferred = getPage ("http://twistedmatrix.com/") # add a callback to the deferred -- request that it run printContents when # the page content has been downloaded deferred.addCallback (printContents) reactor.run()*

3 6

[Twisted-Python] Howto: use Trial to unit test an XML-RPC server
by Don Dwiggins 21 Apr '08

21 Apr '08

I'm offering the following as an experience report and a draft of a howto article: ----------------------------- I've been developing an XML-RPC server using Twisted, and unit testing it with the distribution unittest, with each test connecting as a client and exercising a particular method. I ran into a problem, however: My server is essentially a special-purpose front end to a database, providing limited access to it. Some of the server methods modify the database (I'm using adbapi for DB access). For unit testing purposes, I want each test to leave the test database unchanged when it's done. In unit testing stored procedures, for example, I create a connection in SetUp, use it in the test function to send a SQL "exec", and do a rollback in TearDown. Nice and easy, since I'm using the same connection throughout. In testing my server, however, I don't have access to the connection it used to access the database (and in fact shouldn't have, since my unit test functions are just clients to the server, and know nothing of the database itself). This led me to the following approach, using Twisted's Trial extension of unittest. I converted my unittest module to be run under Trial, as follows: rather than running the server, the test module imports the server module, giving it access to the XMLRPC class itself, and the ability to directly call its methods. Since the server's methods return Deferreds, it's easy enough to call a method, then attach a callback to it that does the checking of its results. Here's an example: ------------- import MyServer class MyServerTests(unittest.TestCase) def setUp(self): # Instantiate object to be tested here self.srvr = MyServer.XMLRPCServer() def testFrobulate(self): d = self.srvr.xmlrpc_frobulate(theFrobulatee) def checkResult(info): # Test that the frobulation occurred correctly pass d.addCallback(checkFrobulation) def checkFrobulation(self, resultOfFrobulation): # Test whether it turned out OK -------------- This works beautifully when frobulate doesn't modify the database; when it does, however, I have the same problem as before: the actual connection used is hidden in the guts of adbapi. For this case, I changed the coding a bit to allow testing. Rather than ConnectionPool.runQuery(), I use .runInteraction(), passing a function that expects a DBAPI cursor as its first argument (when running in the server, the function will be called in the context of a subthread). The code in the server module then looks like this (I'm running on Py 2.4): ------------- @defer.deferredGenerator def xmlrpc_frobulate(theVictim): frobInProgress = defer.waitForDeferred( self._dbpool.runInteraction( self.frobulateInteraction, theVictim) ) yield frobInProgress didItWork = frobInProgress.getResult() yield didItWork return def frobulateInteraction(self, cursor, theVictim): cursor.execute("exec FrobulateOn " + theVictim) # Check the results, return True or False -------------- Now, in the unit test, I can call self.srvr.frobulateInteraction like this: ------------- def testFrobulate(self): self.cursor = self.connection.cursor() d = self.srvr.frobulateInteraction(self.cursor, self.theFrobulatee) def checkResult(info): # Test that the frobulation occurred correctly pass d.addCallback(checkFrobulation) ------------- Now, since frobulateInteraction is using the connection from SetUp, the rollback in TearDown will restore the state of the DB. In effect, by going "under the covers" of the server, I'm bypassing the parts of the server that are supplied by Twisted, and focusing on the code that I've written, which is exactly what I wanted to test. Feedback solicited... -- Don Dwiggins Advanced Publishing Technology

2 2