Mailman 3 May 2006 - Twisted

[Twisted-Python] How many threads in an application
by Christian Schneider 23 May '06

23 May '06

Hi all, I have implemented a server as a subclass of t.i.TCPServer and have it running as a twistd application. Within the service, I have several callbacks. Let's say I have three connected protocols, how many threads to I have? Just one or more. The callbacks are scheduled via reactor.callLater. Thanks, chris

2 2

[Twisted-Python] [ANN] pglib 0.1
by Manlio Perillo 23 May '06

23 May '06

I'm pleasant to announce the first public release of pglib. pglib is an implementation of the PostgreSQL protocol, version 3.0. It is written in Twisted, and aims to be a complete replacement for libpq. In the current version the extended query sub protocol is not supported. The project can be found at: http://developer.berlios.de/projects/pglib/ Subversion access coming soon. Unfortunately, I still have to write some documentation/examples. However there is a (hopefully) detailed README and a complete test case. Its just the "twisted way" ;-). Note that running the test suite requires to setup a PostgreSQL database. And note that pglib offer (at least with the default IRowConsumer implementation) only a raw and low level access to the database (no conversion from SQL types and Python objects, like in pgasync). This is the responsibility of an higher lever layer. I would like to see pglib included in Twisted Core (in twisted.enterprise.protocols.twisted) I'm planning to write a support for MySQL (if I have time, it has a really ugly protocol design) and an asynchronous "wrapper" for SQLite. Regards Manlio Perillo

3 6

[Twisted-Python] Problems with spawnProcess on Windows with ActiveState's ActivePython 2.3
by christian simms 23 May '06

23 May '06

I see this is a known problem (issue #1457 is closed, link at http://twistedmatrix.com/trac/ticket/1457), and the solution is to upgrade your pywin32. However, it's annoying that the current latest ActivePython 2.3 still includes the old and buggy pywin32 -- ActiveState's release notes at http://aspn.activestate.com/ASPN/docs/ActivePython/2.3/relnotes.html state that they are including build 203+ (20050128). As a simple solution, I wonder if someone could add a line to the Trac download link for the "Win32 installer for Python 2.3" saying something like: - NOTE: To avoid issues relating to launching subprocesses, please upgrade to the latest pywin32 package available at: http://sourceforge.net/project/showfiles.php?group_id=78018. If you're using ActiveState's installation then you definitely need to upgrade. Cheers, Christian

1 0

[Twisted-Python] Twisted to proxy php out to apache?
by Remi Cool 23 May '06

23 May '06

Hello fellow twisters, We have a rather large web app based on xul, xmlrpc (via twisted) and php (via apache) .... and we would like to get it all (in due time) in python. As a first step it would be perfect to let twisted handle php page requests so that we can use nevow for the upcomming modules and convert old modules when we have the time to spare. Some time ago, I tried to get PHP to run as a CGI from twisted ... that was so slow that I got timing problems and a non operatable webapp. Nothing new on the FCGI for PHP front yet? My question ... is it possible to pass php page requests from twisted to apache so I don't suffer the speed loss but still have my app listening on only one port (for http and xmlrpc) ? So can twisted be a proxy for php or is this not the recommended way to go? - Remi -

1 0

[Twisted-Python] IPV6 in twisted ?
by Wolfgang Eichler 22 May '06

22 May '06

Hello, what is the situation concerning IPV6 in twisted ? I use tcp LineReceiver. Thanks Wolfgang

2 1

Re: [Twisted-Python] Re: Send many large files with PB
by Brian Warner 22 May '06

22 May '06

> Justin Mazzola Paluska: >> How bad is the slow down? Or, to ask the question another way, how >> much CPU will the process actually take? > > 100% CPU for all the time it takes. Serialization is CPU- and > memory-intensive. Well, the memory footprint is equal to the size of the chunk that you're paging.. that's the reason for using FilePager, to reduce the chunk size. (I think the instantaneous footprint is 2x the chunksize, but once the chunk has been serialized, it drops back to close to 1x). Serializing strings is equivalent to copying them. The banana format for strings is just a couple of length bytes followed by the string contents, so there's not a whole lot of complex CPU stuff going on, just strcpy. The other advantage of FilePager is that the serialization CPU time is spread out according to how fast the network is. I suspect that in most environments, the process will be IO limited, and CPU usage during the process would be far less than 100%. That said, it *is* less efficient than an HTTP server that can just dump the file straight to the network (or better yet use something like sendfile() to avoid the kernel/userspace transition altogether). >> Or is the best thing to do just use the PB to send a URL to the DEST >> server? > > That's what I was hinting at, yes. Of course you should separately take > care of any required authentication, authorization and encryption on the > HTTP connection. Yup, and if you do too much then you're verging back to the performance of PB. Creating a random, unguessable URL which only allows a single download of the target file will basically provide the authentication/authorization features (modulo a man-in-the-middle attack), but not any confidentiality. For some applications that might be enough, though. -Brian

1 0

[Twisted-Python] Calling deferred within a callback, good or bad idea ?
by Stéphane Brault 21 May '06

21 May '06

Hi, I have to call a web service then process the answer, according to the answer I may have to call the service again. The traditional way to go would be : condition = 1 while condition: message = callWebService() condition = processMessage(message) Here is what I do : def function(): def myCalback(message): condition = processMessage(message) if condition: deferred = callWebService() deferred.addCallback(myCallback) deferred = callWebService() deferred.addCallback(myCallback) I was wondering if it was the way to go or if there was a better way, since I'm not quite sure about the impact of calling new deffered within a callback. Thanks, Stéphane

2 2

[Twisted-Python] June Sprint
by Jonathan Lange 21 May '06

21 May '06

Hello! There's a sprint on June 3-4 in Sydney, Australia. But you already knew that. What you don't know is that now there is a wiki page for you to put your name and ideas for the sprint! Examine http://twistedmatrix.com/trac/wiki/SydneySprint Look forward to seeing y'all there. jml

1 1

[Twisted-Python] lore
by Manlio Perillo 21 May '06

21 May '06

Is Lore still the official tool for documentation? I've noted that web2 uses reStructuredText. And what about API generation from docstrings? What tool it is used? epydoc? Thanks and regards Manlio Perillo

7 9

[Twisted-Python] Bugs, bugs, bugs
by Jonathan Lange 21 May '06

21 May '06

Hello, In the interests of household cleanliness[1], I propose we have schedule a weekend of bug squashing. We currently have six hundred and five active tickets. Apart from scheduling a time, and maybe creating a wiki page, I'm not really sure how these things work. I suggest June 3-4 as a good time, as that will match with the sprint in Sydney. What do you reckon? jml [1] http://www.advogato.org/article/215.html

1 0