Efficient debug logging
A common pattern I use is to have logging calls for debug and information with my applications. The logging calls can be separately enabled and disabled. For example: debug_log_enabled = False def debugLog( msg ): If debug_log_enabled: print( ‘Debug: %s’ % (msg,) ) Then the caller can simple write: def main(): debugLog( ‘Start of main’ ) This is fine until the evaluation of the msg becomes expensive. debugLog( ‘info is %r’ % (expensiveFunction(),) ) What would be nice is to be able to avoid evaluation the tuple of arguments if debug is disabled as this can be expensive. I can write this: if debug_log_enabled: debugLog( ‘info is %r’ % (expensiveFunction(),) ) But that is a more code then I would like to write. And if the debug code is a performance problem cannot be left in the production code. I could combine the boolean and the log function by using a class to tidy up the implementation. class DebugLog: def __init__( self, enabled = False ): self.enabled = enabled def __bool__( self ): return self.enabled def __call__( self, msg ): if self.enabled: print( ‘Debug: %s’ % (msg,) ) And call like this: dbg_log = DebugLog() If dbg_log: dbg_log( ‘a debug message’ ) But I’d like to only write: dbg_log( ‘a debug message’ ) And have the evaluation of the argument skipped unless its dbg_log is enabled. I cannot see how to do this with python as it stands. Something would have to be added to allow python to short circuit the argument tuple evaluation. Maybe python can check for a special dunder on the class that know how to do this idiom, __if_true_call__? Thoughts? Barry
Is there any reason logger.isEnabledFor(level), as shown in the docs https://docs.python.org/2/library/logging.html#logging.Logger.isEnabledFor <https://docs.python.org/2/library/logging.html#logging.Logger.isEnabledFor>, is not sufficient for this? Cory
The point is that the cost of creating the msg argument can be very high. At the point that logging decides to skip output it is to late to save the cost of creating the arg tuple. Not that it is relivent for this idea bit logging's levels are too course for logging in complex applications. The app I am working on at the moment has 20 seperate debug categories that are independently enabled. Barry
Cory
The point is that the cost of creating the msg argument can be very high. At the point that logging decides to skip output it is to late to save the cost of creating the arg tuple. This sounds like an optimization that's sufficiently rare and complex to warrant a custom fix or a 3rd party library. Not that it is relivent for this idea bit logging's levels are too course for logging in complete applications. The app I am working on at the moment has 20 seperate debug categories that are independently enabled. Holy balls! That sounds like a tortured use of log levels! On Tue, Feb 14, 2017 at 3:55 PM, Barry <barry@barrys-emacs.org> wrote:
On 2017-02-14 19:51, Abe Dillon wrote:
That's a funny expression. :) I have not got to the point of 20 debug categories in a single file, but I easily have 20+ debug categories in an application. I previously suggested wrapping the expensiveFunction in a lambda, but that is not what I do. I prefer using explicit logging switches: ____if DEBUG_FEATURE_A: ________debugLog(msg, expensiveFunction() ) ____if DEBUG_FEATURE_B: ________debugLog(msg2, expensiveFunction2() ) The benefit is clarity at the expense of more lines, while still avoiding the expensiveFunction() when I can. Using switches result in a smaller log file, because logging is focused on the feature, plus, these switches can still be dynamically set from outside the module. Log "levels" never made sense to me; how can a single dimension be useful substitute for a number of binary switches? With log "levels", you either don't have enough logging, or you drown in too much logging (or you manage a number of loggers, which is worse than logging switches). But to stick to the theme of language features: I believe there is a previous thread that touches on a solution to the original posters problem: "Alternative to PEP 532: delayed evaluation of expressions"
I would like to avoid the if so that the code is more compact if possible.
Log "levels" never made sense to me; how can a single dimension be useful substitute for a number of binary switches? With log "levels", you either don't have enough logging, or you drown in too much logging (or you manage a number of loggers, which is worse than logging switches).
Well said that is indeed the issue with the logger module as is.
But to stick to the theme of language features: I believe there is a previous thread that touches on a solution to the original posters problem: "Alternative to PEP 532: delayed evaluation of expressions
It is not clear to me from one reading is PEP 532 will allow a class with __then__ and __else to do what I want. Would it be something like this: if not debugLog and (expensive(), args()) Does debugLog get passed the tuple in the __then__? Barry
Seems slightly simpler to just make debugLog accept a callable as an alternative to a string. debugLog(lambda:( ‘info is %s’ % expensiveFunction()) ) Op 14 feb. 2017 18:42 schreef "Kyle Lahnakoski" <klahnakoski@mozilla.com>: Can you wrap the expensive functions in lambdas? And have your logger evaluate it, only if required?
debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) )
On 2017-02-14 10:51, Barry Scott wrote:
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 2017-02-14 15:51, Barry Scott wrote:
You could let your debugging function accept a callable and use lambda to delay execution: def debugLog(msg): if debug_log_enabled: if callable(msg): msg = msg() print('Debug: %s' % (msg, )) debugLog('Start of main') debugLog(lambda: 'info is %r' % (expensiveFunction(), ))
There are several mechanisms in the logging module to handle this use-case. First, note that logging functions can take multiple arguments ( https://docs.python.org/3/library/logging.html?highlight=logging#logging.Log... ):
None of the string formatting will take place if the log call is below the level of the logger. It might be worth while to add a class that stores an unevaluated function, then evaluates the function when __str__ or __repr__ is called:
You can also add Filters to your logger: https://docs.python.org/3/library/logging.html?highlight=logging#logging.Fil... On Tue, Feb 14, 2017 at 11:51 AM, MRAB <python@mrabarnett.plus.com> wrote:
On Tue, Feb 14, 2017 at 03:51:05PM +0000, Barry Scott wrote:
Occasionally people (often me... :-) will suggest that it would be good if Python had some sort of lightweight "thunk" for delayed evaluation. I don't have good syntax for that, but let's pretend it was "<# ... #>" just for the sake of having a way to talk about it. We can think of a thunk as being something like syntax for lazy evaluation, so you could write: debugLog( ‘info is %r’ % (<#expensiveFunction()#>, ) ) and the expensive function call would only actually be made if it was required. Python doesn't have thunks, but there is a relatively heavyweight solution for delayed evaluation: wrap the code in a function. debugLog( ‘info is %r’, lambda: expensiveFunction() ) and then adjust debugLog so that if the argument is a function, it will call the function only when needed: def debugLog(message, value): if debug_log_enabled: if isinstance(value, types.FunctionType): value = value() log(message % value)
Indeed. That's what thunks could give us, if only we had a clear picture of how they would work, when they would be evaluated, and what syntax they should use. In the meantime, there are other solutions, not as (hypothetically) lightweight, but they'll work *now* rather than needing to wait for Python 3.7 or 3.9 or 4.5 :-) But if you have some good, concrete suggestions for how to do thunks in Python, please tell us! -- Steve
On Wednesday, 15 February 2017 22:33:42 GMT Chris Angelico wrote:
__repr__ is interesting however. Typically I describe in a string why the value is being logged. That means that the object is always a string. I cannot recall using debugLog( obj ) in production. dlog('This is the state of obj at the start of event processing: %r' % (obj,)) Barry
On Wed, Feb 15, 2017 at 10:47 PM, Barry Scott <barry@barrys-emacs.org> wrote:
Right. I would have no hesitation whatsoever in dumping out a crucial state object, even if it means writing a custom (and expensive) repr function for it. It's clean, simple, and immensely helpful in debugging. ChrisA
On Wednesday, 15 February 2017 22:18:31 GMT Steven D'Aprano wrote:
The lazy eval sound like a generator.
And reply suggested this and it works well for the obviously expensive calls in benchmarks.
At the moment I just put up with the cost of debug calls. The typical of debug messages did come up in a PYCON UK talk and the suggestion was to remove the debug logging after the code works. I prefer to leave the debug logging in production code.
But if you have some good, concrete suggestions for how to do thunks in Python, please tell us!
generators was the best thought so far. Barry
On 15.02.2017, 20:39 Kyle Lahnakoski wrote:
Again, isn't that what Filters are for? I mean the documentation says:
Filters can be used by Handlers and Loggers for more sophisticated filtering than is provided by levels.
On 15.02.2017, 20:39 Kyle Lahnakoski wrote:
That sounds interesting. It sounds a lot like a future. I like that it's a generalization of the problem (lazy expression evaluation) instead of a narrow problem (conditionally logging expensive messages) that is otherwise easily addressed by custom logic. On 16.02.2017, 5:47 Barry Scott wrote:
Loggers can handle that, though. You can easily write:
log.debug("log description: val_1=%r, val_2=%s", 3.14, 100)
Instead of
log.debug("log description: val_1=%r, val_2=%s" % (3.14, 100))
The former won't construct the full message if the log level is higher than debug. It won't repr 3.14 or stringify 100. you can wrap functions in an object that only evaluates them when __repr__ and/or __str__ is called. It doesn't save you much time, because now you're writing a custom class instead of a custom function, I suppose. On Wed, Feb 15, 2017 at 10:46 AM, Sven R. Kunze <srkunze@mail.de> wrote:
Filters could be used instead of a wrapper around the outside of logger.debug() as far as I can see. Buts it not clear that this is better then what I and others seem to do that is wrap the call to logger.debug(). However the use case I’m looking to improve is to avoid the cost of creating the arguments to the logger call.
Agreed if this specific use case is solved by a general mechanism that is fantastic.
Indeed, but it did not stop them all being passed in. And if 3.14 is replaced with calculatePi( places=10000 ) there is no saving. The lambda trick is the best short circuit so far.
you can wrap functions in an object that only evaluates them when __repr__ and/or __str__ is called. It doesn't save you much time, because now you're writing a custom class instead of a custom function, I suppose.
Barry
The default behavior of filters seems pretty useful to me (filtering by hierarchy), but, then again, so do the default log levels. I have trouble understanding when you would need much more, which leads me to believe those cases are probably rare and complex (though I might very well be wrong). If you have a complex problem you typically should expect a complex solution like writing a custom log filter. You might think "it would be nice if the solution to this problem was in the standard lib", but we have to be a little picky about what goes into the standard lib least it inflate to a monstrous mess. If it's a niche problem, maybe a custom solution or third party lib *is* the way to go. and may also be non-local There are design patterns to help with that. and still does not solve the expensiveFunction() problem. I wasn't trying to address that problem in that quote. I was addressing the tangent on the insufficient expressiveness of log levels. I personally don't see why you can't use floats for log levels, but I'm interested to know how people are using logs such that they need dozens of levels. That, however; is tangential to the discussion about conditional execution of an expensive function. On Thu, Feb 16, 2017 at 12:54 PM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:
On 16 Feb 2017, at 21:03, Abe Dillon <abedillon@gmail.com> wrote:
I personally don't see why you can't use floats for log levels, but I'm interested to know how people are using logs such that they need dozens of levels. That, however; is tangential to the discussion about conditional execution of an expensive function.
Its not lots of levels its lots of categories. I did add one level, INFOHEADER, but not for debug. It you are interested in what I did you could look at this project: https://github.com/barry-scott/scm-workbench Have a look at: https://github.com/barry-scott/scm-workbench/blob/master/Source/Scm/wb_scm_d... https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_de... Then look at almost any module and see the use of the _debug calls. Barry
So if python converted: debugLog( <# ‘format string %r’ % (expensive(),) #> ) Into: def __tmp__(): yield ‘format string %r’ % (expensive(),) debugLog( __tmp__ ) Then debugLog can detect the generator and call __next__ only if logging is enabled. I guess at that point its the same as using the lambda but you have the syntax sugar of Steven’s <# … #>. Barry
Yeah, I had a similar issue in a previous company. A colleague wrote a script using a regex to remove these debug logs in the .py code. IHMO the clean design for that would be to support officially preprocessors in Python. My PEP opens the gate for that: https://www.python.org/dev/peps/pep-0511/ It would allow to write easily your own light preprocessor just to remove debug logs. Victor Le 14 févr. 2017 5:24 PM, "Barry Scott" <barry@barrys-emacs.org> a écrit :
Some comments: 1. you don't need a preprocessor for this: simply put your logging code into an "if __debug__:" block: https://docs.python.org/3.6/reference/simple_stmts.html?the-assert-statement... and then run your production code with "python -O" (the trick here is that the Python byte code compiler will not even generate code for such ifs) 2. from experience, keeping logging active in production often outweighs the benefits of the little better performance you gain by disabling it - being able to fix a failed app run by replaying transactions based on log file content can be priceless 3. preprocessors are evil, let's please not have them in Python :-) Code example for 1: app.py ------ def test(): assert 1 == 0, "assert raises" if __debug__: raise ValueError('__debug__ run branch') print ('test complete') test()
On 16.02.2017 13:26, Victor Stinner wrote:
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 16 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
2017-02-16 13:55 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
1. you don't need a preprocessor for this: simply put your logging code into an "if __debug__:" block:
The problem with -O is that it also disables assertions, whereas you may want to keep them at runtime on production for good reasons. Victor
On 16.02.2017 14:23, Victor Stinner wrote:
Assertions that you need in production should be written as proper if-statements. I know some people will disagree, but IMO using "assert" is the wrong approach in such situations - it's meant for development and testing only, not as short-cut to avoid having to write a proper error handler :-) If they were, Python would not drop creating code for them when using Python in production -O mode. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 16 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
Is there any reason logger.isEnabledFor(level), as shown in the docs https://docs.python.org/2/library/logging.html#logging.Logger.isEnabledFor <https://docs.python.org/2/library/logging.html#logging.Logger.isEnabledFor>, is not sufficient for this? Cory
The point is that the cost of creating the msg argument can be very high. At the point that logging decides to skip output it is to late to save the cost of creating the arg tuple. Not that it is relivent for this idea bit logging's levels are too course for logging in complex applications. The app I am working on at the moment has 20 seperate debug categories that are independently enabled. Barry
Cory
The point is that the cost of creating the msg argument can be very high. At the point that logging decides to skip output it is to late to save the cost of creating the arg tuple. This sounds like an optimization that's sufficiently rare and complex to warrant a custom fix or a 3rd party library. Not that it is relivent for this idea bit logging's levels are too course for logging in complete applications. The app I am working on at the moment has 20 seperate debug categories that are independently enabled. Holy balls! That sounds like a tortured use of log levels! On Tue, Feb 14, 2017 at 3:55 PM, Barry <barry@barrys-emacs.org> wrote:
On 2017-02-14 19:51, Abe Dillon wrote:
That's a funny expression. :) I have not got to the point of 20 debug categories in a single file, but I easily have 20+ debug categories in an application. I previously suggested wrapping the expensiveFunction in a lambda, but that is not what I do. I prefer using explicit logging switches: ____if DEBUG_FEATURE_A: ________debugLog(msg, expensiveFunction() ) ____if DEBUG_FEATURE_B: ________debugLog(msg2, expensiveFunction2() ) The benefit is clarity at the expense of more lines, while still avoiding the expensiveFunction() when I can. Using switches result in a smaller log file, because logging is focused on the feature, plus, these switches can still be dynamically set from outside the module. Log "levels" never made sense to me; how can a single dimension be useful substitute for a number of binary switches? With log "levels", you either don't have enough logging, or you drown in too much logging (or you manage a number of loggers, which is worse than logging switches). But to stick to the theme of language features: I believe there is a previous thread that touches on a solution to the original posters problem: "Alternative to PEP 532: delayed evaluation of expressions"
I would like to avoid the if so that the code is more compact if possible.
Log "levels" never made sense to me; how can a single dimension be useful substitute for a number of binary switches? With log "levels", you either don't have enough logging, or you drown in too much logging (or you manage a number of loggers, which is worse than logging switches).
Well said that is indeed the issue with the logger module as is.
But to stick to the theme of language features: I believe there is a previous thread that touches on a solution to the original posters problem: "Alternative to PEP 532: delayed evaluation of expressions
It is not clear to me from one reading is PEP 532 will allow a class with __then__ and __else to do what I want. Would it be something like this: if not debugLog and (expensive(), args()) Does debugLog get passed the tuple in the __then__? Barry
Seems slightly simpler to just make debugLog accept a callable as an alternative to a string. debugLog(lambda:( ‘info is %s’ % expensiveFunction()) ) Op 14 feb. 2017 18:42 schreef "Kyle Lahnakoski" <klahnakoski@mozilla.com>: Can you wrap the expensive functions in lambdas? And have your logger evaluate it, only if required?
debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) )
On 2017-02-14 10:51, Barry Scott wrote:
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
On 2017-02-14 15:51, Barry Scott wrote:
You could let your debugging function accept a callable and use lambda to delay execution: def debugLog(msg): if debug_log_enabled: if callable(msg): msg = msg() print('Debug: %s' % (msg, )) debugLog('Start of main') debugLog(lambda: 'info is %r' % (expensiveFunction(), ))
There are several mechanisms in the logging module to handle this use-case. First, note that logging functions can take multiple arguments ( https://docs.python.org/3/library/logging.html?highlight=logging#logging.Log... ):
None of the string formatting will take place if the log call is below the level of the logger. It might be worth while to add a class that stores an unevaluated function, then evaluates the function when __str__ or __repr__ is called:
You can also add Filters to your logger: https://docs.python.org/3/library/logging.html?highlight=logging#logging.Fil... On Tue, Feb 14, 2017 at 11:51 AM, MRAB <python@mrabarnett.plus.com> wrote:
On Tue, Feb 14, 2017 at 03:51:05PM +0000, Barry Scott wrote:
Occasionally people (often me... :-) will suggest that it would be good if Python had some sort of lightweight "thunk" for delayed evaluation. I don't have good syntax for that, but let's pretend it was "<# ... #>" just for the sake of having a way to talk about it. We can think of a thunk as being something like syntax for lazy evaluation, so you could write: debugLog( ‘info is %r’ % (<#expensiveFunction()#>, ) ) and the expensive function call would only actually be made if it was required. Python doesn't have thunks, but there is a relatively heavyweight solution for delayed evaluation: wrap the code in a function. debugLog( ‘info is %r’, lambda: expensiveFunction() ) and then adjust debugLog so that if the argument is a function, it will call the function only when needed: def debugLog(message, value): if debug_log_enabled: if isinstance(value, types.FunctionType): value = value() log(message % value)
Indeed. That's what thunks could give us, if only we had a clear picture of how they would work, when they would be evaluated, and what syntax they should use. In the meantime, there are other solutions, not as (hypothetically) lightweight, but they'll work *now* rather than needing to wait for Python 3.7 or 3.9 or 4.5 :-) But if you have some good, concrete suggestions for how to do thunks in Python, please tell us! -- Steve
On Wednesday, 15 February 2017 22:33:42 GMT Chris Angelico wrote:
__repr__ is interesting however. Typically I describe in a string why the value is being logged. That means that the object is always a string. I cannot recall using debugLog( obj ) in production. dlog('This is the state of obj at the start of event processing: %r' % (obj,)) Barry
On Wed, Feb 15, 2017 at 10:47 PM, Barry Scott <barry@barrys-emacs.org> wrote:
Right. I would have no hesitation whatsoever in dumping out a crucial state object, even if it means writing a custom (and expensive) repr function for it. It's clean, simple, and immensely helpful in debugging. ChrisA
On Wednesday, 15 February 2017 22:18:31 GMT Steven D'Aprano wrote:
The lazy eval sound like a generator.
And reply suggested this and it works well for the obviously expensive calls in benchmarks.
At the moment I just put up with the cost of debug calls. The typical of debug messages did come up in a PYCON UK talk and the suggestion was to remove the debug logging after the code works. I prefer to leave the debug logging in production code.
But if you have some good, concrete suggestions for how to do thunks in Python, please tell us!
generators was the best thought so far. Barry
On 15.02.2017, 20:39 Kyle Lahnakoski wrote:
Again, isn't that what Filters are for? I mean the documentation says:
Filters can be used by Handlers and Loggers for more sophisticated filtering than is provided by levels.
On 15.02.2017, 20:39 Kyle Lahnakoski wrote:
That sounds interesting. It sounds a lot like a future. I like that it's a generalization of the problem (lazy expression evaluation) instead of a narrow problem (conditionally logging expensive messages) that is otherwise easily addressed by custom logic. On 16.02.2017, 5:47 Barry Scott wrote:
Loggers can handle that, though. You can easily write:
log.debug("log description: val_1=%r, val_2=%s", 3.14, 100)
Instead of
log.debug("log description: val_1=%r, val_2=%s" % (3.14, 100))
The former won't construct the full message if the log level is higher than debug. It won't repr 3.14 or stringify 100. you can wrap functions in an object that only evaluates them when __repr__ and/or __str__ is called. It doesn't save you much time, because now you're writing a custom class instead of a custom function, I suppose. On Wed, Feb 15, 2017 at 10:46 AM, Sven R. Kunze <srkunze@mail.de> wrote:
Filters could be used instead of a wrapper around the outside of logger.debug() as far as I can see. Buts it not clear that this is better then what I and others seem to do that is wrap the call to logger.debug(). However the use case I’m looking to improve is to avoid the cost of creating the arguments to the logger call.
Agreed if this specific use case is solved by a general mechanism that is fantastic.
Indeed, but it did not stop them all being passed in. And if 3.14 is replaced with calculatePi( places=10000 ) there is no saving. The lambda trick is the best short circuit so far.
you can wrap functions in an object that only evaluates them when __repr__ and/or __str__ is called. It doesn't save you much time, because now you're writing a custom class instead of a custom function, I suppose.
Barry
The default behavior of filters seems pretty useful to me (filtering by hierarchy), but, then again, so do the default log levels. I have trouble understanding when you would need much more, which leads me to believe those cases are probably rare and complex (though I might very well be wrong). If you have a complex problem you typically should expect a complex solution like writing a custom log filter. You might think "it would be nice if the solution to this problem was in the standard lib", but we have to be a little picky about what goes into the standard lib least it inflate to a monstrous mess. If it's a niche problem, maybe a custom solution or third party lib *is* the way to go. and may also be non-local There are design patterns to help with that. and still does not solve the expensiveFunction() problem. I wasn't trying to address that problem in that quote. I was addressing the tangent on the insufficient expressiveness of log levels. I personally don't see why you can't use floats for log levels, but I'm interested to know how people are using logs such that they need dozens of levels. That, however; is tangential to the discussion about conditional execution of an expensive function. On Thu, Feb 16, 2017 at 12:54 PM, Kyle Lahnakoski <klahnakoski@mozilla.com> wrote:
On 16 Feb 2017, at 21:03, Abe Dillon <abedillon@gmail.com> wrote:
I personally don't see why you can't use floats for log levels, but I'm interested to know how people are using logs such that they need dozens of levels. That, however; is tangential to the discussion about conditional execution of an expensive function.
Its not lots of levels its lots of categories. I did add one level, INFOHEADER, but not for debug. It you are interested in what I did you could look at this project: https://github.com/barry-scott/scm-workbench Have a look at: https://github.com/barry-scott/scm-workbench/blob/master/Source/Scm/wb_scm_d... https://github.com/barry-scott/scm-workbench/blob/master/Source/Common/wb_de... Then look at almost any module and see the use of the _debug calls. Barry
So if python converted: debugLog( <# ‘format string %r’ % (expensive(),) #> ) Into: def __tmp__(): yield ‘format string %r’ % (expensive(),) debugLog( __tmp__ ) Then debugLog can detect the generator and call __next__ only if logging is enabled. I guess at that point its the same as using the lambda but you have the syntax sugar of Steven’s <# … #>. Barry
Yeah, I had a similar issue in a previous company. A colleague wrote a script using a regex to remove these debug logs in the .py code. IHMO the clean design for that would be to support officially preprocessors in Python. My PEP opens the gate for that: https://www.python.org/dev/peps/pep-0511/ It would allow to write easily your own light preprocessor just to remove debug logs. Victor Le 14 févr. 2017 5:24 PM, "Barry Scott" <barry@barrys-emacs.org> a écrit :
Some comments: 1. you don't need a preprocessor for this: simply put your logging code into an "if __debug__:" block: https://docs.python.org/3.6/reference/simple_stmts.html?the-assert-statement... and then run your production code with "python -O" (the trick here is that the Python byte code compiler will not even generate code for such ifs) 2. from experience, keeping logging active in production often outweighs the benefits of the little better performance you gain by disabling it - being able to fix a failed app run by replaying transactions based on log file content can be priceless 3. preprocessors are evil, let's please not have them in Python :-) Code example for 1: app.py ------ def test(): assert 1 == 0, "assert raises" if __debug__: raise ValueError('__debug__ run branch') print ('test complete') test()
On 16.02.2017 13:26, Victor Stinner wrote:
-- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 16 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
2017-02-16 13:55 GMT+01:00 M.-A. Lemburg <mal@egenix.com>:
1. you don't need a preprocessor for this: simply put your logging code into an "if __debug__:" block:
The problem with -O is that it also disables assertions, whereas you may want to keep them at runtime on production for good reasons. Victor
On 16.02.2017 14:23, Victor Stinner wrote:
Assertions that you need in production should be written as proper if-statements. I know some people will disagree, but IMO using "assert" is the wrong approach in such situations - it's meant for development and testing only, not as short-cut to avoid having to write a proper error handler :-) If they were, Python would not drop creating code for them when using Python in production -O mode. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Experts (#1, Feb 16 2017)
::: We implement business ideas - efficiently in both time and costs ::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ http://www.malemburg.com/
participants (15)
-
Abe Dillon -
Barry -
Barry Scott
-
Barry Warsaw -
Chris Angelico -
Cory Benfield -
Giampaolo Rodola' -
Kyle Lahnakoski -
M.-A. Lemburg -
MRAB -
Stephan Houben -
Stephen J. Turnbull -
Steven D'Aprano -
Sven R. Kunze -
Victor Stinner