Mailman 3 February 2007 - Twisted

Re: [Twisted-Python] questions about large db queries and memory leak
by slowtech chen Feb. 9, 2007

Feb. 9, 2007

From: Thomas Hervé <therve(a)free.fr> >I guess you use adbapi ? The performance are generally not a problem with db >queries because most of the job is done on the DB, not on the client. I've got >servers that makes hundreds requests against Postgres/MySQL/sqlite without >problems. Yes, I will use adbapi and MySQL, good to hear that. regards, Chen ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com

1 0

[Twisted-Python] Support for ICMP ping?
by Nicolas D. Cesar Feb. 8, 2007

Feb. 8, 2007

I've been googling if there is any 'ICMP ping' support in Twisted. I've encounter ancient replies: http://twistedmatrix.com/pipermail/twisted-python/2003-August/005542.html I wonder if today is any ICMP support. I've tried to search the source tree[1], and doesn't seem. My actual needs is creating a DeferredList with "ping-testers" for different IPs, just to check if there up or not. I know I can end up using twisted.internet.utils.getProcessOutput and calling ping, as I would normally do from command line. For other newbies as me in the same situation please read: http://twistedmatrix.com/users/moshez/talk.html and search for 'getProcessOutput' ;-) Greetings, [1] my search was: -------------------------------------------------------------------- $ grep -C3 -ir icmp * internet/protocol.py- internet/protocol.py- internet/protocol.py-class AbstractDatagramProtocol: internet/protocol.py: """Abstract protocol for datagram-oriented transports, e.g. IP, ICMP, ARP, UDP.""" internet/protocol.py- internet/protocol.py- transport = None internet/protocol.py- numPorts = 0 -- internet/protocol.py- def connectionRefused(self): internet/protocol.py- """Called due to error from write in connected mode. internet/protocol.py- internet/protocol.py: Note this is a result of ICMP message generated by *previous* internet/protocol.py- write. internet/protocol.py- """ internet/protocol.py- -------------------------------------------------------------------- -- Nicolás D. César <ncesar(a)lunix.com.ar> Lunix S.R.L. -[ http://www.lunix.com.ar ]- GnuPG Public Key: gpg --keyserver wwwkeys.pgp.net --recv-key 0x3606F3E6

5 7

Re: [Twisted-Python] Something strange about cred
by glyph＠divmod.com Feb. 8, 2007

Feb. 8, 2007

On 03:16 pm, stephen.c.waterbury(a)nasa.gov wrote: >P.S. Sorry I can't irc from work -- NASA/GSFC IT policy >doesn't allow that. :( Clearly you're allowed to exchange messages with dangerous subversives like us, so would it help if we wrote an IRC-via-SMTP proxy? ;-)

3 2

[Twisted-Python] startLogging
by Martin Evans Feb. 8, 2007

Feb. 8, 2007

Are there any samples on how best to interact with logging? My server needs to log quite a few items, as such the log file is growing quite quickly and I would like to be able to switch file names on a daily basis. I would also probably then wish to delete logs older than 14 days. I'm not sure how safe it would be to just keep calling startLogging() with a different file name mid-session. I realise there has been some discussion about this area recently. Thanks, Martin

6 5

[Twisted-Python] wxreactor, serialport, and Windows
by Andy.Henshaw＠gtri.gatech.edu Feb. 8, 2007

Feb. 8, 2007

I have some twisted code that supports forwarding serial port data to a tcp socket. The code needs to run on Windows, so I'm using the win32eventreactor, as is required by the twisted.internet.serialport code. Now, I need to marry it to a wxPython GUI. Is there any reasonable way to support the twisted.internet.serialport code using wxreactor?

5 6

[Twisted-Python] questions about large db queries and memory leak
by slowtech chen Feb. 8, 2007

Feb. 8, 2007

Hey guys, I've been using twisted for a couple days, I like it very much. And there are some stuffs I 'm still not quite get: - I like the asyn way twisted provides, but when using RDBMS, I have to spawn other threads out of the main event loop. this seems weird and I 'm wondering how the performance would be when you've got a lot of concurrent db queries. - Did anyone come into memory leak on twisted server? 'cause when I tested our twisted server today ,the memory usage became 75% when idle. Any suggestion welcome. regards, Chen ____________________________________________________________________________________ No need to miss a message. Get email on-the-go with Yahoo! Mail for Mobile. Get started. http://mobile.yahoo.com/mail

3 2

Re: [Twisted-Python] Re: Using twisted.trial with nose unittest system
by glyph＠divmod.com Feb. 8, 2007

Feb. 8, 2007

On 05:36 pm, antoine.pitrou(a)wengo.com wrote: >> More serious than the process-running issue, from my point of view, is >> the fact that if you forget the "@deferred" decorator on a test that >> uses any reactor API, or code that internally _uses_ any reactor API, >> you will put your tests into a non-deterministic state, and although >> they'll pass _most_ of the time, they will die depending on the load >> characteristics of your machine at the time and (insert long-winded >> rant about the evil things threads do to your code here). Tests are >> exactly the place you never want this to happen :). > >That's certainly right, although you should get messages about uncaught >exceptions or unhandled errbacks, so you'll know that something went >wrong. That's not the sort of problem I'm talking about. Some reactor APIs will appear to work when called from the wrong thread, so if you forget a @deferred somewhere, your test might seem to work, or hang in a weird place, or produce different error output each time. >What happens with Trial when you forget to return the Deferred the test >should wait on? Trial doesn't require tests to return Deferreds. If there is no Deferred, the test completes immediately. >> All in all it seems like it would be simpler to just use trials' >> TestCase class with nose. Is there any reason not to? > >Given that Trial is a self-contained tool, and given that nose also has >its own test running and collecting, I simply didn't expect this to work >or to keep working accross versions of both tools. This is definitely a concern, but your current approach doesn't seem any less fragile. >> Trial doesn't force your code to make use of Twisted, and as Mr. Lange >> has declared previously, it goes to some lengths to be compatible with >> stdlib unit tests. > >Of course, but to use Trial you must install Twisted don't you ? >Which is quite a big dependency, Is 2 megabytes really too big? >and Twisted does not yet support being easy_install'ed. ... and nose doesn't yet support being apt-get installed ;-) If you'd like to see easy_install support for Twisted, please feel free to contribute support for it; the release manager could probably use some help maintaining setup.py in general.

3 2

Re: [Twisted-Python] amptypes 0.1
by glyph＠divmod.com Feb. 8, 2007

Feb. 8, 2007

On 07:58 pm, thostrup(a)gmail.com wrote: >Hi > >I've created a couple of extension types for the AMP protocol. Thanks! Would you consider contributing these to Twisted proper? Aside from a few very minor cleanups (test naming convention, a few missing docstrings) these look great.

2 1

Re: [Twisted-Python] Re: Using twisted.trial with nose unittest system
by glyph＠divmod.com Feb. 7, 2007

Feb. 7, 2007

On 09:53 pm, mithrandi-twisted-python(a)mithrandi.za.net wrote: >* glyph(a)divmod.com <glyph(a)divmod.com> [2007-02-07 21:24:18 -0000]: > >> ... and nose doesn't yet support being apt-get installed ;-) > >mithrandi@elvardein:~> sudo apt-get install python-nose Well! Egg on my face, there. Where's the apt line?

2 1

Re: [Twisted-Python] Re: Using twisted.trial with nose unittest system
by glyph＠divmod.com Feb. 7, 2007

Feb. 7, 2007

On 03:32 pm, antoine.pitrou(a)wengo.com wrote: > >Hi Jean-Paul, > >> The one practical problem which this method of integration has is that >> it will not allow unit tests to run child processes. So, one wouldn't >> be able to run Twisted's test suite with nose using this. Of course, >> if one has no interest in running tests which run child processes, then >> this may not be a big deal. > >Do you mean because the reactor does not receive signals? Yes. Also, it's highly likely that the tests won't stop cleanly on ^C. More serious than the process-running issue, from my point of view, is the fact that if you forget the "@deferred" decorator on a test that uses any reactor API, or code that internally _uses_ any reactor API, you will put your tests into a non-deterministic state, and although they'll pass _most_ of the time, they will die depending on the load characteristics of your machine at the time and (insert long-winded rant about the evil things threads do to your code here). Tests are exactly the place you never want this to happen :). The documentation for this module should be clear on the fact that Twisted-using tests are _absolutely required_ to use the decorator, even if they don't need to wait on a deferred. Of course, that implies that the "you must return a twisted Deferred" message should probably be removed and that case dealt with. Also, setUp and tearDown methods also need the decorator applied, I think, not just test methods. All in all it seems like it would be simpler to just use trials' TestCase class with nose. Is there any reason not to? >> >Let me give a few points as for nose vs. Trial : You are welcome to use nose, of course, but some of these reasons sounded kind of flimsy to me. >> >- nose does not depend on Twisted. Unless all your Python projects make >> >use of Twisted, you certainly don't like having to use a different >> >testing tool depending on the project. Trial doesn't force your code to make use of Twisted, and as Mr. Lange has declared previously, it goes to some lengths to be compatible with stdlib unit tests. It can run and discover stdlib tests, as well as being run by the stdlib runner, and I've used it to test tiny projects which do not use Twisted code anywhere. So, Trial only requires Twisted insofar as Nose requires Nose. It's part of its implementation. If you use Twisted for _any_ of your development, it's already installed. >> >- nose has nice features, including the ability to write tests as >> >functions without having to write classes This just sounds like TIMTOWTDI to me. I *like* the fact that all test methods look the same ;). >> >- nose has a plugin system which already provides several extensions In the spirit of completeness, see trial's --help-reporters and --reporter option. These are discovered using plugins. Please do not take this argument seriously though. :) There is widespread consensus that this plugin API is not very good and it is likely to change. Still, it exists and you can use it. See: http://twistedmatrix.com/trac/browser/sandbox/exarkun/merit/trunk/twisted/p… Although, perhaps the utility of the plugin system is decreased more by the fact that nobody knows about it than by its sub-optimal API, which leads into your next point... >> >- nose is documented This is a very, very good point. Trial's documentation is extremely weak. Does anyone (and by "anyone" I mean "anyone but JML, who has already done more work on trial than any reasonable person ever would") want to volunteer to write a "test driven development with Twisted" lore document? I've introduced many people to Twisted in recent weeks, and each time I ended up having to do a live demonstration of how to write a test module, how to run the trial command line, and (for those lucky enough to use emacs) how to get twisted-dev.el loaded so that f9 will do the right thing. This stuff isn't clear even to people already familiar with Python and unit testing, let alone to people who are learning TDD for the first time with trial.

4 3