-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/30/2011 10:39 AM, Andrew Bennetts wrote:
John Arbash Meinel wrote: […]
Just for the record, changing cleanFailure to not do its operation is likely to lead to big memory leaks. The second thing you did, setting self.tb to None, makes more sense to speed things up.
Right. I agree that it is a better solution. I was wondering about having a whitelist of Exception classes that aren't considered interesting enough to keep a traceback.
I've filed <http://twistedmatrix.com/trac/ticket/5011> about this issue, and I have a patch that takes an alternative approach: don't capture tracebacks when Deferred callbacks raise exceptions unless defer.setDebugging(True) has been called.
Another possibility might be to stop Failure from capturing all the frame details (i.e. the locals and globals) by default (or at least when invoked by Deferred._runCallbacks), as that's probably the bulk of the expense and would be very rarely used.
-Andrew.
I think walking the frames and copying the dicts is also expensive. That is what the bug you linked to me was about. (First, walking everything and using __dict__.copy() was a bit expensive, and second that the safe_repr() calls were turning 1GB strings into a new 1GB+ string.) The one other step that I think we need, is that 'maybeDeferred' also always traps into a Failure object, and we'd want that to check Deferred.debug first. I do wonder if Failure should just be checking Deferred.debug before automatically including a traceback. I'm not really sure about logical layering of twisted modules, though. Certainly 'twisted.python.failure' seems a lower layer than 'twisted.internet.defer'. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2S7uAACgkQJdeBCYSNAAPS9ACfdGmt9td2jqOFDZYY6mJy/6DN UiMAnRF0w9pID0WMP0MMGwh9LSQN9RqM =g3j7 -----END PGP SIGNATURE-----