[Python-Dev] A new way to configure logging

Vinay Sajip vinay_sajip at yahoo.co.uk
Wed Oct 7 16:49:44 CEST 2009


At present, configuration of Python's logging package can be done in one of two
ways:

1. Create a ConfigParser-readable configuration file and use
logging.config.fileConfig() to read and implement the configuration therein.
2. Use the logging API to programmatically configure logging using getLogger(),
addHandler() etc.

The first of these works for simple cases but does not cover all of the logging
API (e.g. Filters). The second of these provides maximal control, but besides
requiring users to write the configuration code, it fixes the configuration in
Python code and does not facilitate changing it easily at runtime.

In addition, the ConfigParser format appears to engender dislike (sometimes
strong dislike) in some quarters. Though it was chosen because it was the only
configuration format supported in the stdlib at that time, many people regard it
(or perhaps just the particular schema chosen for logging's configuration) as
'crufty' or 'ugly', in some cases apparently on purely aesthetic grounds. Recent
versions of Python of course support an additional battery-included format which
can be used for configuration - namely, JSON. Other options, such as YAML, are
also possible ways of configuring systems, Google App Engine-style, and PyYAML
has matured nicely.

There has also been talk on the django-dev mailing list about providing better
support for using Python logging in Django. When it happens (as of course I hope
it does) this has the consequence that many new users who use Django but are
relatively inexperienced in Python (e.g. in PHP shops which are moving to
Django) will become exposed to Python logging. As Django is configured using a
Python module and use of ConfigParser-style files is not a common approach in
that ecosystem, users will find either of the two approaches outlined above a
particular pain point when configuring logging for their Django applications and
websites, unless something is done to avoid it.

All three of the contenders for the title of "commonly found configuration
mechanism" - JSON, YAML and Python code - will be expressible, in Python, as
Python dicts. So it seems to make sense to add, to logging.config, a new
callable bound to "dictConfig" which will take a single dictionary argument and
configure logging from that dictionary.

An important facet of implementing such a scheme will be the format or schema
which the dictionary has to adhere to. I have started working on what such a
schema would look like, and if people here think it's a good idea to go ahead
with this, I'll provide the details of the schema on a separate post which I'll
also cross-post on comp.lang.python so that I can get feedback from there, too.

In outline, the scheme I have in mind will look like this, in terms of the new
public API:

class DictConfigurator:
    def __init__(self, config): #config is a dict-like object (duck-typed)
        import copy
        self.config = copy.deepcopy(config)
        
    def configure(self):
        #  actually do the configuration here using self.config

dictConfigClass = DictConfigurator

def dictConfig(config):
    dictConfigClass(config).configure()

This allows easy replacement of DictConfigurator with a suitable subclass where
needed.

What's the general feeling here about this proposal? All comments and
suggestions will be gratefully received.

Regards,


Vinay Sajip




More information about the Python-Dev mailing list