logging.config.defaultConfig()
Using logging (as library) is easy in Python. But setting up logging is done different in nearly every environment. I would like to have a common way to **load** the configuration. The configuration itself should be done in the environment the scripts runs in. If you write a console script, and you want it to be reusable in different environments, there is no default way at the moment (or at least I don't see any) to let the environment set up the logging. I want a standard hook: The console script should be able to call into the surrounding environment. This improves reusablity. I know how to use dictConfig() or fileConfig(), but these methods need parameters. And what the parameters look like should not be defined in the reusable console script. I think the following solution is very flexible and solves most needs to set up logging, since I can implement your needs in for example your_environment_module.set_up() {{{ def defaultConfig(): ''' Load module to set_up() the logging configuration of your environment. Reads the module name from: os.environ.get('LOGGINGCONFIG', 'loggingconfig') Calls set_up() on the imported module. Would be nice to have this as logging.config.defaultConfig Related: https://docs.python.org/2/library/logging.config.html ''' module_name = os.environ.get('LOGGINGCONFIG', 'loggingconfig') module = importlib.import_module(module_name) module.set_up() }}} Do you understand what I propose? What do you think? Thomas Güttler -- Thomas Güttler http://thomas-guettler.de/
On Fri, 16 May 2014 09:05:11 +0200 Thomas Güttler <guettli@thomas-guettler.de> wrote:
I think the following solution is very flexible and solves most needs to set up logging, since I can implement your needs in for example your_environment_module.set_up()
This looks dubious to me. There is no reason to have a shared Python logging configuration, IMO. Also, I don't understand why this is importing a module. If all your scripts are part of an application, then it's reasonable for them to share a mechanism for logging configuration. But it should be done in your application, not in Python itself. Regards Antoine.
On 16.05.2014 11:27, Antoine Pitrou wrote:
While I agree that importing a module might not be the right way, having a standard way to configure logging via environment variables might be helpful. Configuring logging is a difficult thing if done fully, like, allowing different loglevels for different loggers. Having this implemented in the standard library might be actually useful (and it’s also done that way in other languages). regards, Jonas
On Fri, 16 May 2014 11:32:55 +0200 Jonas Wielicki <j.wielicki@sotecware.net> wrote:
I entirely disagree. An environment variable is a very lousy way to specify a configuration file's location; and there is no reason to have a common logging configuration for all Python applications.
What does this have to do with environment variables? logging.dictConfig() already does this. Regards Antoine.
Am 16.05.2014 11:54, schrieb Antoine Pitrou:
** I don't want a common logging configuration ** I want a standard hook to find the logging configuration. And I want it to be a Python method. If you prefer a file config, create a method which loads your config file. This would make the spec "simple and stupid". The configuration should be empty by default. Only if the environment wants to have a common config, it should provide one. Thomas -- Thomas Guettler http://www.thomas-guettler.de/
On Fri, 16 May 2014 13:08:17 +0200 Thomas Guettler <guettli@thomas-guettler.de> wrote:
Why would that be Python's business? If the hook is meant to be truly "standard", then it should be something like a LSB standard. End users don't really care whether some application is written in Python or another language. Why a Python-specific hook? What do users gain?
And I want it to be a Python method.
Basically you are telling us what /you/ want, but not why it would be useful for the broader community. Regards Antoine.
On 16.05.2014 11:54, Antoine Pitrou wrote:
Hmm, it's a fairly standard way to define config file locations esp. on Unix platforms, so I don't follow you here. Perhaps I'm just missing some context. Such env vars are often used in application environments to override system defaults, e.g. for finding OpenSSL or ODBC config files. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2014)
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 16 May 2014 21:34, M.-A. Lemburg <mal@egenix.com> wrote:
Python is a language runtime, not an application. Having globally configurable behaviours for a runtime is, in general, questionable, which is why we have the options to ignore the environment variables, site-packages, user site-packages and now the "isolated mode" flag that basically says "ignore *every* explicitly configurable Python setting in the environment". For 3.2+, we defined a sensible default logging configuration (warning and above written to stderr, everything else ignored), so users should be free to just use the logging module when writing libraries without worrying about whether or not it has been configured for reporting properly. That doesn't help Python 2 users, but that's the case for a lot of things. Trying to provide a way to actually *configure* logging in a general way would be fraught with backwards compatibility issues when it came to interfering with frameworks (whether for writing CLI applications, web applications, or GUI applications) that already providing their own way of handling logging configuration. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 17.05.2014 10:56, schrieb Nick Coghlan:
Using logging as library works well in Python. But writing console scripts which use logging force the developer to solve the same problems again and again: How to set up the logging? And the developer of the console script does not know how the user of the console script wants to handle logging. That's why all Python application have a different way to set up the logging.
Of course a standard way to get the logging configuration defined by the end user should be optional. I don't see any backwards compatibility issues. The author of the console script should just need one line, to get the defaults which the console script user wants. {{{ import argparse def main() logging.config.defaultConfig() argparse... }}} The end user can set up the logging in the way he wants: - Log to a file - Log to a daemon - Format the messages the way he likes it - ... Since I know that some logging environments are complicated, I think it is best to hook into a method call. There are environments where fileConfig() does not solve all needs. Please ask if, you don't understand what I want. Thomas Güttler -- Thomas Güttler http://thomas-guettler.de/
Am 17.05.2014 14:08, schrieb Antoine Pitrou:
If you want to get the config from a database or LDAP. Is this supported by fileConfig()? -- Thomas Güttler http://thomas-guettler.de/
On Sat, 17 May 2014 14:41:05 +0200 Thomas Güttler <guettli@thomas-guettler.de> wrote:
Obviously not, but you should be able to use dictConfig() for that. Mapping the database contents to the dict representation expected by dictConfig() is a domain-specific task that cannot be provided by the standard library, so it's the application's job to provide it. Regards Antoine.
On 17 May 2014 22:05, "Thomas Güttler" <guettli@thomas-guettler.de> wrote:
But this is also why command line frameworks like Cement exist (disclosure: Cement was just the first example I found of the kind of full featured CLI framework I mean. There may be other examples). I guess my question is, if an application is to the point of worrying about configuring logging, why should we handle it in the interpreter for command line applications, when we leave it to frameworks to handle for web and GUI applications? Application configuration is a complicated problem - you have to decide what to do about global defaults, user defaults, environment variables, command line options, potentially runtime adjustable options for daemons, SIGHUP handling, etc, etc. This complexity, along with other questions like ini-format vs JSON vs YAML is a key part of *why* PEP 391 punted on the question and just defined logging.dictConfig() instead. Cheers, Nick.
end user
On 17.05.2014 10:56, Nick Coghlan wrote:
Right, but those options address specific use cases (e.g. for setting up testing environments). Their existence does not imply that having config variables for all of the above is a bad thing, as you seem to imply - otherwise, we wouldn't have them in the first place ;-) Logging is just another runtime feature, just like writing PYC files or setting a search path. Now, configuring logging is too complex to do on the command line, so pointing the runtime to a logging config file instead seems like a good idea. Of course, an application could just as well do this, so the question really is whether we should have it in general or not. PS: Note that with "application environment" I'm referring to exactly that: a shell environment with environment options specifically setup for a specific application. You typically use those for application specific user accounts, not globally. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 17 2014)
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On Fri, 16 May 2014 09:05:11 +0200 Thomas Güttler <guettli@thomas-guettler.de> wrote:
I think the following solution is very flexible and solves most needs to set up logging, since I can implement your needs in for example your_environment_module.set_up()
This looks dubious to me. There is no reason to have a shared Python logging configuration, IMO. Also, I don't understand why this is importing a module. If all your scripts are part of an application, then it's reasonable for them to share a mechanism for logging configuration. But it should be done in your application, not in Python itself. Regards Antoine.
On 16.05.2014 11:27, Antoine Pitrou wrote:
While I agree that importing a module might not be the right way, having a standard way to configure logging via environment variables might be helpful. Configuring logging is a difficult thing if done fully, like, allowing different loglevels for different loggers. Having this implemented in the standard library might be actually useful (and it’s also done that way in other languages). regards, Jonas
On Fri, 16 May 2014 11:32:55 +0200 Jonas Wielicki <j.wielicki@sotecware.net> wrote:
I entirely disagree. An environment variable is a very lousy way to specify a configuration file's location; and there is no reason to have a common logging configuration for all Python applications.
What does this have to do with environment variables? logging.dictConfig() already does this. Regards Antoine.
Am 16.05.2014 11:54, schrieb Antoine Pitrou:
** I don't want a common logging configuration ** I want a standard hook to find the logging configuration. And I want it to be a Python method. If you prefer a file config, create a method which loads your config file. This would make the spec "simple and stupid". The configuration should be empty by default. Only if the environment wants to have a common config, it should provide one. Thomas -- Thomas Guettler http://www.thomas-guettler.de/
On Fri, 16 May 2014 13:08:17 +0200 Thomas Guettler <guettli@thomas-guettler.de> wrote:
Why would that be Python's business? If the hook is meant to be truly "standard", then it should be something like a LSB standard. End users don't really care whether some application is written in Python or another language. Why a Python-specific hook? What do users gain?
And I want it to be a Python method.
Basically you are telling us what /you/ want, but not why it would be useful for the broader community. Regards Antoine.
On 16.05.2014 11:54, Antoine Pitrou wrote:
Hmm, it's a fairly standard way to define config file locations esp. on Unix platforms, so I don't follow you here. Perhaps I'm just missing some context. Such env vars are often used in application environments to override system defaults, e.g. for finding OpenSSL or ODBC config files. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 16 2014)
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
On 16 May 2014 21:34, M.-A. Lemburg <mal@egenix.com> wrote:
Python is a language runtime, not an application. Having globally configurable behaviours for a runtime is, in general, questionable, which is why we have the options to ignore the environment variables, site-packages, user site-packages and now the "isolated mode" flag that basically says "ignore *every* explicitly configurable Python setting in the environment". For 3.2+, we defined a sensible default logging configuration (warning and above written to stderr, everything else ignored), so users should be free to just use the logging module when writing libraries without worrying about whether or not it has been configured for reporting properly. That doesn't help Python 2 users, but that's the case for a lot of things. Trying to provide a way to actually *configure* logging in a general way would be fraught with backwards compatibility issues when it came to interfering with frameworks (whether for writing CLI applications, web applications, or GUI applications) that already providing their own way of handling logging configuration. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Am 17.05.2014 10:56, schrieb Nick Coghlan:
Using logging as library works well in Python. But writing console scripts which use logging force the developer to solve the same problems again and again: How to set up the logging? And the developer of the console script does not know how the user of the console script wants to handle logging. That's why all Python application have a different way to set up the logging.
Of course a standard way to get the logging configuration defined by the end user should be optional. I don't see any backwards compatibility issues. The author of the console script should just need one line, to get the defaults which the console script user wants. {{{ import argparse def main() logging.config.defaultConfig() argparse... }}} The end user can set up the logging in the way he wants: - Log to a file - Log to a daemon - Format the messages the way he likes it - ... Since I know that some logging environments are complicated, I think it is best to hook into a method call. There are environments where fileConfig() does not solve all needs. Please ask if, you don't understand what I want. Thomas Güttler -- Thomas Güttler http://thomas-guettler.de/
Am 17.05.2014 14:08, schrieb Antoine Pitrou:
If you want to get the config from a database or LDAP. Is this supported by fileConfig()? -- Thomas Güttler http://thomas-guettler.de/
On Sat, 17 May 2014 14:41:05 +0200 Thomas Güttler <guettli@thomas-guettler.de> wrote:
Obviously not, but you should be able to use dictConfig() for that. Mapping the database contents to the dict representation expected by dictConfig() is a domain-specific task that cannot be provided by the standard library, so it's the application's job to provide it. Regards Antoine.
On 17 May 2014 22:05, "Thomas Güttler" <guettli@thomas-guettler.de> wrote:
But this is also why command line frameworks like Cement exist (disclosure: Cement was just the first example I found of the kind of full featured CLI framework I mean. There may be other examples). I guess my question is, if an application is to the point of worrying about configuring logging, why should we handle it in the interpreter for command line applications, when we leave it to frameworks to handle for web and GUI applications? Application configuration is a complicated problem - you have to decide what to do about global defaults, user defaults, environment variables, command line options, potentially runtime adjustable options for daemons, SIGHUP handling, etc, etc. This complexity, along with other questions like ini-format vs JSON vs YAML is a key part of *why* PEP 391 punted on the question and just defined logging.dictConfig() instead. Cheers, Nick.
end user
On 17.05.2014 10:56, Nick Coghlan wrote:
Right, but those options address specific use cases (e.g. for setting up testing environments). Their existence does not imply that having config variables for all of the above is a bad thing, as you seem to imply - otherwise, we wouldn't have them in the first place ;-) Logging is just another runtime feature, just like writing PYC files or setting a search path. Now, configuring logging is too complex to do on the command line, so pointing the runtime to a logging config file instead seems like a good idea. Of course, an application could just as well do this, so the question really is whether we should have it in general or not. PS: Note that with "application environment" I'm referring to exactly that: a shell environment with environment options specifically setup for a specific application. You typically use those for application specific user accounts, not globally. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, May 17 2014)
::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
participants (6)
-
Antoine Pitrou -
Jonas Wielicki -
M.-A. Lemburg -
Nick Coghlan -
Thomas Guettler -
Thomas Güttler