[Python-ideas] Efficient debug logging
mal at egenix.com
Thu Feb 16 07:55:47 EST 2017
1. you don't need a preprocessor for this: simply put your
logging code into an "if __debug__:" block:
and then run your production code with "python -O"
(the trick here is that the Python byte code compiler will
not even generate code for such ifs)
2. from experience, keeping logging active in production often
outweighs the benefits of the little better performance you
gain by disabling it - being able to fix a failed app run
by replaying transactions based on log file content can be
3. preprocessors are evil, let's please not have them in
Code example for 1:
assert 1 == 0, "assert raises"
raise ValueError('__debug__ run branch')
print ('test complete')
> python3 -O
Python 3.4.5 (default, Nov 17 2016, 20:58:13)
[GCC 4.8.1 20130909 [gcc-4_8-branch revision 202388]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> import app
Disassembly of test:
5 0 LOAD_GLOBAL 0 (print)
3 LOAD_CONST 1 ('test complete')
6 CALL_FUNCTION 1 (1 positional, 0 keyword pair)
10 LOAD_CONST 0 (None)
On 16.02.2017 13:26, Victor Stinner wrote:
> Yeah, I had a similar issue in a previous company. A colleague wrote a
> script using a regex to remove these debug logs in the .py code.
> IHMO the clean design for that would be to support officially preprocessors
> in Python. My PEP opens the gate for that:
> It would allow to write easily your own light preprocessor just to remove
> debug logs.
> Le 14 févr. 2017 5:24 PM, "Barry Scott" <barry at barrys-emacs.org> a écrit :
>> A common pattern I use is to have logging calls for debug and information
>> with my applications.
>> The logging calls can be separately enabled and disabled.
>> For example:
>> debug_log_enabled = False
>> def debugLog( msg ):
>> If debug_log_enabled:
>> print( ‘Debug: %s’ % (msg,) )
>> Then the caller can simple write:
>> def main():
>> debugLog( ‘Start of main’ )
>> This is fine until the evaluation of the msg becomes expensive.
>> debugLog( ‘info is %r’ % (expensiveFunction(),) )
>> What would be nice is to be able to avoid evaluation the tuple of
>> arguments if debug is
>> disabled as this can be expensive. I can write this:
>> if debug_log_enabled: debugLog( ‘info is %r’ %
>> (expensiveFunction(),) )
>> But that is a more code then I would like to write. And if the debug code
>> is a performance problem cannot
>> be left in the production code.
>> I could combine the boolean and the log function by using a class to tidy
>> up the implementation.
>> class DebugLog:
>> def __init__( self, enabled = False ):
>> self.enabled = enabled
>> def __bool__( self ):
>> return self.enabled
>> def __call__( self, msg ):
>> if self.enabled: print( ‘Debug: %s’ % (msg,) )
>> And call like this:
>> dbg_log = DebugLog()
>> If dbg_log: dbg_log( ‘a debug message’ )
>> But I’d like to only write:
>> dbg_log( ‘a debug message’ )
>> And have the evaluation of the argument skipped unless its dbg_log is
>> I cannot see how to do this with python as it stands.
>> Something would have to be added to allow python to short circuit the
>> argument tuple evaluation.
>> Maybe python can check for a special dunder on the class that know how to
>> do this idiom, __if_true_call__?
>> Python-ideas mailing list
>> Python-ideas at python.org
>> Code of Conduct: http://python.org/psf/codeofconduct/
> Python-ideas mailing list
> Python-ideas at python.org
> Code of Conduct: http://python.org/psf/codeofconduct/
Professional Python Services directly from the Experts (#1, Feb 16 2017)
>>> Python Projects, Coaching and Consulting ... http://www.egenix.com/
>>> Python Database Interfaces ... http://products.egenix.com/
>>> Plone/Zope Database Interfaces ... http://zope.egenix.com/
::: We implement business ideas - efficiently in both time and costs :::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
More information about the Python-ideas