Mailman 3 August 2006 - Twisted

[Twisted-Python] Reducing memory footprint of twisted app?
by Marcin Kasperski 17 Aug '06

17 Aug '06

Not an exact question, but rather 'searching for ideas'. I have some twisted app, which uses more memory than I would like it to. I tried analysing it a bit (mainly using gc module object list and enumerating items of different types) and it seem to me that there is something 'twistedish' in it. My application uses in a lot of places generator idiom (functions/methods which yield wrapped with defer.deferredGenerator). And, as there seem to be a lot of anonymous functions and tuples allocated, I suspect that maybe those functions, deferreds and related params and closures live longer then I would like to. Any ideas of what could I do to track it down? In particular, is it possible to somehow use introspection to find which lambdas and deferreds are allocated while the program is running? Are there any suggestions on how to code deferredGenerators to reduce allocated memory (maybe, for instance, I should try to turn local variables into object attributes, or opposite, or ...) Also, if anybody could me point to any interesting resources about tracking python momory usage, I would be grateful. Tried googling for some time, but apart of zope trackRef I did not found anything.

4 6

[Twisted-Python] using stdin with twisted
by Norman Harman 17 Aug '06

17 Aug '06

I have a program that gets apache log on stdin. I would like to add a web interface and some RPC stuff to it. I imagine something like protocols.basic.LineReceiver is what I want. But I can't figure out what ClientFactory to use or ??? I can find very little info on doing this. The stdio package is depreciated and the stdin.py example does not run. thanks, norm

2 2

[Twisted-Python] Producers / Consumers and more examples tutorials? (Example Request)
by Andrew Smith 17 Aug '06

17 Aug '06

>Just name a specific example you'd like to see and I'll try to write >something up. I don't want to pick something, write it up, and then >discover that it doesn't cover what you wanted to learn. :) >Jean-Paul How about this scenario: I make an HTTP request and expect to receive an XML document back. The XML document is a container for discreet items. It might look something like this: <items> <item> <content>foo</content> </item> <item> <content>bar</content> </item> </items> I don't know how big this document will be or how long it will take to retrieve so I would like to present myself as a producer of individual item elements and let clients consume them as they become available. Does this fit the twisted Producer/Consumer paradigm? -a. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

[Twisted-Python] Killing a process
by Igor Kravtchenko 17 Aug '06

17 Aug '06

Hi, I wondered last time what is the best way under Win32 to kill a process launched with: reactor.spawnProcess() os.kill() as documented here: http://docs.python.org/lib/os-process.html#l2h-1676 only works under Macintosh and Unix. Furthermore, spawnProcess() doesn't return any handle, process ID or anything to deal with. Any help or idea would be really appreciated, Igor.

4 4

[Twisted-Python] will the real dave peticolas please stand up
by glyph＠divmod.com 17 Aug '06

17 Aug '06

In an attempt to clean out some old cruft from the tracker, we're starting to play a virtual game of "hot potato", highlighting the least responsive developer in the Twisted dev team, by using this report: http://twistedmatrix.com/trac/report/13 It's currently a fixture of the IRC channel topic; maybe we'll make it more interactive in the future. Currently the winner (or, rather, the loser) is Dave Peticolas, the closest thin to a maintainer that twisted.enterprise has. We haven't heard from him in quite a while. If you're still on this list, Mr. Peticolas, would you please do Andrew Bennetts the honor of passing the target of our collective ridicule to him by updating ticket #303? :) If you're no longer interested in working on Twisted we'll start reassigning those tickets still left in the tracker to others.

2 2

[Twisted-Python] problems with tracebacks
by Manlio Perillo 17 Aug '06

17 Aug '06

This problem seems to happen from time in time. Traceback are not printed. Now I'm running trial on a test case and I got: [...] test.test_mail.TestMail.test_email_plain ... [OK] test.test_mail.TestMail.test_sendmail_html ... [ERROR] test.test_mail.TestMail.test_sendmail_text ... [ERROR] =============================================================================== [ERROR]: test.test_mail.TestMail.test_sendmail_html Failure: exceptions.IndexError: list index out of range =============================================================================== [ERROR]: test.test_mail.TestMail.test_sendmail_text Failure: twisted.internet.defer.TimeoutError: <test.test_mail.TestMail testMetho d=test_sendmail_text> (test_sendmail_text) still running at 3.0 secs ------------------------------------------------------------------------------- Ran 4 tests in 3.078s FAILED (errors=2, successes=1) And the log file: [...] 2006/08/04 21:59 ora solare Europa occidentale [-] --> test.test_mail.TestMail.test_email_plain <-- 2006/08/04 21:59 ora solare Europa occidentale [-] --> test.test_mail.TestMail.test_sendmail_html <-- 2006/08/04 21:59 ora solare Europa occidentale [-] --> test.test_mail.TestMail.test_sendmail_text <-- 2006/08/04 21:59 ora solare Europa occidentale [-] Main loop terminated. 2006/08/04 21:59 ora solare Europa occidentale [-] Main loop terminated. Unfortunately every time I got this problem I'm unable to write a small reproducible code. And every time the problem is caused by a simple bug in the code. P.S Why I got the "Main loop terminated" two times? Thanks and regards Manlio Perillo

4 6

[Twisted-Python] Is Twisted adequate for usage in reactive robots?
by Willi Richert 15 Aug '06

15 Aug '06

Hi Twisted developers, for our soccer robot we are evaluating moving away from Java for the high level behavior code in parts to Python. At the moment we have a bunch of Java threads that listen at the various comm. channels for incoming and outgoing messages and pass them to the main Java processing thread. We plan to reimplement all the basic behavior stuff (move to, avoid, ...) that returns abstract movement commands in Python. For that the Python process has to communicate with a bunch of processes (particle filter, etc.) using our own robotics optimized message format (the same format is used to communicate using TCP/IP, CAN, serial). After I've read a bunch of success stories I thought that maybe Twisted could be used to abstract all the gory low-level network details. Would that be advisable for a highly reactive robot? The main loop of the Python process would have to run at 50Hz. Longer taking calculation process are processed in a separate thread. Is there any comparison regarding asynchronous and thread execution/latency/... speeds? Regards, wr

2 1

[Twisted-Python] IResource and http.Site()
by Jon Parise 14 Aug '06

14 Aug '06

I'm relatively new to Twisted, so apologies if I'm missing something obvious. I'm using Twisted 2.4, and I'm trying to use twisted.web with my own resource implementation. It looks like my options are to either derive from http.Resource or implement the http.IResource interface directly. I've decided to start with the latter. The IResource interface declares three methods: getChildWithDefault(self, name, request) putChild(self, path, child) render(self, request) However, parts of http.Site() (such as getResourceFor()) expect my object to offer additional methods (such as getChildForRequest()) which are not part of the interface. My question is therefore: Is http.Site() only compatible with Resource (and not IResource), or should the IResource interface include some additional methods, such as getChildForRequest()? Thanks! -- Jon Parise (jon of csh.rit.edu) :: http://www.csh.rit.edu/~jon/

1 2

[Twisted-Python] spawnProcess under Win32
by Igor Kravtchenko 14 Aug '06

14 Aug '06

Hi! I had recently to use the Process protocol of Twisted (win32eventreactor) to launch some applications and grab their output. It works well, the only problem I have is Twisted always opens a DOS console. Since I receive the output and treat them myself, the DOS console stays empty. But in fact, I would never want this console to appear on screen. I've looked at the way Twisted creates a process under Win32 and I've noticed that it uses the CreateProcess() function. The problem is it doesn't mention any creation flags (i.e.: it just passes 0) whereas here, we would need to pass DETACHED_PROCESS to avoid a new console to be created. What I've done is merely to add an optional win32flags argument to the spawnProcess() method in win32eventreactor.py as follow: http://cvs.gna.org/cvsweb/envwin32/python/Lib/site-packages/twisted/interne… I would be interested to know what do you think about this? Does it seem to you a good idea? Sincerely, Igor.

5 6

[Twisted-Python] Using FQDN with transport.write calls...
by Chaz. 13 Aug '06

13 Aug '06

My application requires that I use FQDN in place of the normal IP addresses in transport.write calls. In looking over I see quite a few places called 'resolveAddress', which look like they do exactly what I need. I was wondering if anyone has a code snippet that would show me exactly what I need to do? TIA, Chaz

5 8