On Mon, Mar 28, 2011 at 9:44 AM, John Arbash Meinel <john@arbash-meinel.com>wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm doing some performance testing of one of our Twisted applications. And what I came across was a surprising amount of time being spent in twisted.python.failure.Failure.__getstate__
So under profiling, we spent 408ms in __getstate__. I then changed Failure.cleanFailure to just 'return', and I saw a real-world improvement of ~480ms down to about 240ms. I then restored cleanFailure, but changed Failure.__init__ to always set 'tb=None' before it does anything. And in that case the time went down to 180ms. (And when I dug into it, 150ms of that is time spent waiting for an XMLRPC response.)
Just for the record, changing cleanFailure to not do its operation is likely to lead to big memory leaks. The second thing you did, setting self.tb to None, makes more sense to speed things up.
I'm wondering if there is a tasteful way to disable Traceback processing in a production machine. I realize you never know when you are going to need the state in order to figure out what went wrong. But it is costing 2-5x runtime speed. (The other answer is to write all of our code to avoid Deferred.addErrback...)
I hate to mention it (since it's unlikely I'd be able to work on it), but I wonder how much a C extension would improve this without going the full step of throwing out the debug data. -- Christopher Armstrong http://radix.twistedmatrix.com/ http://planet-if.com/