Logging into single file from multiple modules in python when TimedRotatingFileHandler is used
Chethan Kumar S
chethu.kumar88 at gmail.com
Tue Jun 21 05:04:52 EDT 2022
Hi all,
Need help with below query on python logging module.
I have a main process which makes use of different other modules. And these modules also use other modules. I need to log all the logs into single log file. Due to use of TimedRotatingFileHandler, my log behaves differently after midnight. I got to know why it is so but couldn't get how I can solve it.
Issue was because of serialization in logging when multiple processes are involved.
Below is log_config.py which is used by all other modules to get the logger and log.
import logging
import sys
from logging.handlers import TimedRotatingFileHandler
FORMATTER = logging.Formatter("%(asctime)s — %(name)s — %(message)s")
LOG_FILE = "my_app.log"
def get_file_handler():
file_handler = TimedRotatingFileHandler(LOG_FILE, when='midnight')
file_handler.setFormatter(FORMATTER)
return file_handler
def get_logger(logger_name):
logger = logging.getLogger(logger_name)
logger.setLevel(logging.DEBUG) # better to have too much log than not enough
logger.addHandler(get_file_handler())
#with this pattern, it's rarely necessary to propagate the error up to parent
logger.propagate = False
return logger
All other modules call, 'logging = log_config.get_logger(name)' and use it to log.
I came to know about QueueHandler and QueueListener but not sure how to use them in my code. How can I use these to serialize logs to single file.?
Any help is appreciated.
Thanks,
Chethan
More information about the Python-list
mailing list