Multiprocessing & Logging

[Vinay Sajip]

Thibaut <merwin.irc <at> gmail.com> writes:

> This is exactly what I wanted, it seems perfect. However I still have a 
> question, from what I understood,
> I have to configure logging AFTER creating the process, to avoid 
> children process to inherits the logging config.
> 
> Unless there is a way to "clean" logging configuration in children 
> processes, so they only have one handler : the QueueHandler.
> 
> I looked at the logging code and it doesn't seems to have an easy way to 
> do this. The problem of configuring the logging
> after the process creation is that... I can't log during process 
> creation. But if it's too complicated, I will just do this.

You may be able to have a "clean" configuration: for example, dictConfig()
allows the configuration dictionary to specify whether existing loggers are
disabled. So the details depend on the details of your desired configuration.

One more point: I suggested that you subclass QueueListener, but you don't
actually need to do this. For example, you can do something like:

class DelegatingHandler(object):
    def handle(self, record):
        logger = logging.getLogger(record.name)
        logger.handle(record)

And then instantiate the QueueListener with an instance of DelegatingHandler.
QueueListener doesn't need actual logging handlers, just something with a handle
method which takes a record.

Regards,

Vinay Sajip