Re : [Twisted-Python] twisted performance and reactor choice
Thanks Jean-Paul, my mistake ;-) I must confess that if I use twisted (and I must say I'm quite happy with it), I didn't have time to know it that well. Anyway, I was wondering if changing the reactor could help a little (my application runs on a debian sarge 3.1). In which case would the pollreactor be relevant or should I stick with the default one (selectreactor if I read the documentation right). I have a lot of simultaneous deferred at scheduled times (both database access using zalchemy and connections to external sites which can take some times). I know this might not be enough to make a decision, but any help would be welcomed. Thanks again, Stéphane ----- Message d'origine ---- De : Jean-Paul Calderone <exarkun@divmod.com> À : Twisted general discussion <twisted-python@twistedmatrix.com> Envoyé le : Mardi, 14 Novembre 2006, 18h03mn 31s Objet : Re: [Twisted-Python] twisted performance and reactor choice On Tue, 14 Nov 2006 09:14:39 +0000 (GMT), Stéphane Brault <stephane_brault@yahoo.fr> wrote:
Hi,
My application has lots of concurrent connections, both in and out, since it communicates with outsides sites. It seems that it is sometimes slowed down, I guess because there are to many simultaneous deffered. I'd like to know if I could switch reactor (my application is accessed by XML-RPC and communicates with external applications through XML-RPC, Http requests and access ftp sites). I also need to know how large can be the threadpool in order to increase the number of simultaneous deferred.
You've gone a bit too fast here. You guessed at the cause of the slowdown. Instead, you need to profile the application and find out what the cause really is. Your question about the size of the threadpool is a bit of a non sequitur. The threadpool, let alone its size, is unrelated to how many Deferreds might exist in your application at once. Unless you are explicitly using the threadpool (or doing a significant number of DNS lookups without using Twisted Names), the threadpool isn't even related to how many concurrent tasks might be in progress in your application. This is because Twisted only uses the threadpool for an extremely limited number of purposes. Basic network traffic isn't one of these purposes. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com
On Tue, 14 Nov 2006 17:51:04 +0000 (GMT), Stéphane Brault <stephane_brault@yahoo.fr> wrote:
Thanks Jean-Paul,
my mistake ;-) I must confess that if I use twisted (and I must say I'm quite happy with it), I didn't have time to know it that well.
No problem. :) Optimization is hard.
Anyway, I was wondering if changing the reactor could help a little (my application runs on a debian sarge 3.1). In which case would the pollreactor be relevant or should I stick with the default one (selectreactor if I read the documentation right).
If the application is actually bottlenecked in select, then it might help a little. Fortunately, it's easy to test. Just run: twistd --reactor poll ... Or if you're not using twistd, add: from twisted.internet import pollreactor pollreactor.install() To the beginning of your application. That's all that's necessary to change to a different reactor.
I have a lot of simultaneous deferred at scheduled times (both database access using zalchemy and connections to external sites which can take some times). I know this might not be enough to make a decision, but any help would be welcomed.
Changing the reactor won't speed up Deferreds or threads, though, since all the reactors have the same thread code. Jean-Paul
participants (2)
-
Jean-Paul Calderone
-
Stéphane Brault