PYTHONLOGGING env variable
Another idea I've had that may be of use: PYTHONLOGGING environment variable. Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level. Another option would be that -x dev or a different -x logging will initialize basic config. Will be useful mostly for debugging purposes instead of temporarily modifying the code. Kinda surprised it doesn't exist tbh. Bar Harel
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example: # assuming loglevel is bound to the string value obtained from the# command line argument. Convert to upper case to allow the user to# specify --log=DEBUG or --log=debugnumeric_level = getattr(logging, loglevel.upper(), None)if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel)logging.basicConfig(level=numeric_level, ...) (https://docs.python.org/3/howto/logging.html#logging-to-a-file) Is there a practical reason why the above doesn't work for you? Otherwise, I see no reason to add a new environment variable that effectively does the same thing. On Wed, Feb 19, 2020 at 8:16 PM Bar Harel <bzvi7919@gmail.com> wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
Will be useful mostly for debugging purposes instead of temporarily modifying the code.
Kinda surprised it doesn't exist tbh.
Bar Harel _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/I74LVJ... Code of Conduct: http://python.org/psf/codeofconduct/
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
Note: "loglevel" is an arbitrary name, but this can be implemented trivially with something like argparse to check for a --log argument. On Wed, Feb 19, 2020 at 9:19 PM Kyle Stanley <aeros167@gmail.com> wrote:
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
# assuming loglevel is bound to the string value obtained from the# command line argument. Convert to upper case to allow the user to# specify --log=DEBUG or --log=debugnumeric_level = getattr(logging, loglevel.upper(), None)if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel)logging.basicConfig(level=numeric_level, ...)
(https://docs.python.org/3/howto/logging.html#logging-to-a-file)
Is there a practical reason why the above doesn't work for you? Otherwise, I see no reason to add a new environment variable that effectively does the same thing.
On Wed, Feb 19, 2020 at 8:16 PM Bar Harel <bzvi7919@gmail.com> wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
Will be useful mostly for debugging purposes instead of temporarily modifying the code.
Kinda surprised it doesn't exist tbh.
Bar Harel _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/I74LVJ... Code of Conduct: http://python.org/psf/codeofconduct/
Thanks Kyle for the feedback :-) Unfortunately, there is no logging.loglevel attribute. At the moment, the only way of doing this is modifying the program, parsing input parameters and setting up the logging system. If you want it only temporarily for debugging purposes, or permanently for all scripts in your system, it's not really feasible. Think of -Wdefault. You can change settings in the warnings module and modify files each time you wish to enable all warnings. But then again, why would you? We have debugging flags for warnings, faulthandlers, asyncio etc... Logging I would say is the most important and trivial of all, yet we don't have a flag for it. On Thu, Feb 20, 2020, 4:24 AM Kyle Stanley <aeros167@gmail.com> wrote:
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
Note: "loglevel" is an arbitrary name, but this can be implemented trivially with something like argparse to check for a --log argument.
On Wed, Feb 19, 2020 at 9:19 PM Kyle Stanley <aeros167@gmail.com> wrote:
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
# assuming loglevel is bound to the string value obtained from the# command line argument. Convert to upper case to allow the user to# specify --log=DEBUG or --log=debugnumeric_level = getattr(logging, loglevel.upper(), None)if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel)logging.basicConfig(level=numeric_level, ...)
(https://docs.python.org/3/howto/logging.html#logging-to-a-file)
Is there a practical reason why the above doesn't work for you? Otherwise, I see no reason to add a new environment variable that effectively does the same thing.
On Wed, Feb 19, 2020 at 8:16 PM Bar Harel <bzvi7919@gmail.com> wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
Will be useful mostly for debugging purposes instead of temporarily modifying the code.
Kinda surprised it doesn't exist tbh.
Bar Harel _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/I74LVJ... Code of Conduct: http://python.org/psf/codeofconduct/
[re-sending this since I forgot to CC the list]
Unfortunately, there is no logging.loglevel attribute.
At the moment, the only way of doing this is modifying the program, parsing input parameters and setting up the logging system.
Indeed, that's why I clarified in the follow-up message; sorry it's been a long day :-). "loglevel" was simply an arbitrary name chosen for that particular example and getattr() was used as a way to prefix the module so it would access the logging level constant instead of the literal name of the level as a string (which would be invalid to pass to level). You'd likely use argparse or some other command-line argument parsing utility to get it in the first place. For example: parser = argparse.ArgumentParser() parser.add_argument("--log", dest='logging_level', default='WARNING') args = parser.parse_args() numeric_level = getattr(logging, args.logging_level.upper()) logging.basicConfig(level=numeric_level, ...) (input validation omitted for simplification)
If you want it only temporarily for debugging purposes, or permanently for all scripts in your system, it's not really feasible.
Hmm, I'm not sure that I like the idea of overriding the default logging level being set system-wide, or that there would be an especially strong use case for it. Instead, it seems more practically common to want to set it on an individual run of a program temporarily. In such cases, using some form of command-line parsing utility is a perfectly a viable option.
We have debugging flags for warnings, faulthandlers, asyncio etc... Logging I would say is the most important and trivial of all, yet we don't have a flag for it.
Arguably though, the easier it is to implement on your own, the less of a need there is to be an environment variable to do it. Also, I think the logging configuration should be on a per-program basis, rather than globally (which is what env vars are typically used for). FWIW I'm not opposed to the principle behind the idea, I'm just not convinced yet that it would be that much better than the options that are already available, and would be worthwhile to add an entirely new environment variable for (or be good usage of an env var for that matter). Perhaps there are other areas to explore for simplifying the process of changing the logging level from the command-line though. Also, although not the same proposal, there was a discussion that brought up some similar points in a python-ideas thread back in 2014: https://mail.python.org/archives/list/python-ideas@python.org/thread/7ZLMSQU.... I think some of the points made in there are relevant to this discussion. On Wed, Feb 19, 2020 at 10:26 PM Bar Harel <bzvi7919@gmail.com> wrote:
Thanks Kyle for the feedback :-)
Unfortunately, there is no logging.loglevel attribute.
At the moment, the only way of doing this is modifying the program, parsing input parameters and setting up the logging system.
If you want it only temporarily for debugging purposes, or permanently for all scripts in your system, it's not really feasible.
Think of -Wdefault. You can change settings in the warnings module and modify files each time you wish to enable all warnings. But then again, why would you?
We have debugging flags for warnings, faulthandlers, asyncio etc... Logging I would say is the most important and trivial of all, yet we don't have a flag for it.
On Thu, Feb 20, 2020, 4:24 AM Kyle Stanley <aeros167@gmail.com> wrote:
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
Note: "loglevel" is an arbitrary name, but this can be implemented trivially with something like argparse to check for a --log argument.
On Wed, Feb 19, 2020 at 9:19 PM Kyle Stanley <aeros167@gmail.com> wrote:
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
This can already be done from the command line through --log=LEVEL, which sets the logging.loglevel attribute. In the how-to section of the docs, this is demonstrated in the following example:
# assuming loglevel is bound to the string value obtained from the# command line argument. Convert to upper case to allow the user to# specify --log=DEBUG or --log=debugnumeric_level = getattr(logging, loglevel.upper(), None)if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s' % loglevel)logging.basicConfig(level=numeric_level, ...)
(https://docs.python.org/3/howto/logging.html#logging-to-a-file)
Is there a practical reason why the above doesn't work for you? Otherwise, I see no reason to add a new environment variable that effectively does the same thing.
On Wed, Feb 19, 2020 at 8:16 PM Bar Harel <bzvi7919@gmail.com> wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
Will be useful mostly for debugging purposes instead of temporarily modifying the code.
Kinda surprised it doesn't exist tbh.
Bar Harel _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/I74LVJ... Code of Conduct: http://python.org/psf/codeofconduct/
If you want it only temporarily for debugging purposes, or permanently for all scripts in your system, it's not really feasible.
Hmm, I'm not sure that I like the idea of overriding the default logging level being set system-wide, or that there would be an especially strong use case for it. Instead, it seems more practically common to want to set it on an individual run of a program temporarily. In such cases, using some form of command-line parsing utility is a perfectly a viable option.
Well I meant more like a permanent "debug" mode, but the temporary case is probably the one more appealing to me and other users.
We have debugging flags for warnings, faulthandlers, asyncio etc... Arguably though, the easier it is to implement on your own, the less of a need there is to be an environment variable to do it. Also, I think the logging configuration should be on a per-program basis, rather than globally (which is what env vars are typically used for).
I agree that logging configuration should be on a per program basis. Maybe a flag should be the solution, like '-Linfo'. When I said 'trivial', I didn't mean easy to implement. I meant that it's a much more common use case, and makes sense to have. -Wdefault is trivial to implement. Asyncio.debug is a single flag. Logging takes the same amount of lines, even more if you need to parse the args, but is much more common and needed. I've done a little stack overflow search, and there are tons of questions about having or creating such a feature manually. Last but not least: our suggested methodology for libraries is to use "NullHandler". If a library uses it, errors will not show up unless you set up a handler (with probably stdout for debug) on the root logger. If '-Linfo' or '-x logging' will be part of the possible debug cmdline options, these obscure bugs would be much easier to track. (I personally spent a few days to track one like that not long ago)
On 2020-02-19 17:12, Bar Harel wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
As a heavy user of logging, I like this one a lot. In a new project, the lack of such a feature allows one to stick with print() perhaps longer than is prudent. -Mike
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system? But a standard command line argument to the interpreter could be useful. -CHB On Thu, Feb 20, 2020 at 8:51 AM Mike Miller <python-ideas@mgmiller.net> wrote:
On 2020-02-19 17:12, Bar Harel wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
As a heavy user of logging, I like this one a lot. In a new project, the lack of such a feature allows one to stick with print() perhaps longer than is prudent.
-Mike _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Let's plan a cmdline argument then. Should we go for -L<level> much like -W<warning>? Or -x logging for setting a default value of debug or info? Per python specification, -x is implementation dependent. I'll advocate for -L. I think the same concerns as -W are relevant for all python implementations, and it gives more flexibility in terms of log level. Next question: Should -x dev include it? If it was python day-1, I would say yes. My issue is the double inclusion of stream handlers in case the logging was bootstrapped already. So -x dev probably shouldn't include it. On Thu, Feb 20, 2020, 8:28 PM Christopher Barker <pythonchb@gmail.com> wrote:
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system?
But a standard command line argument to the interpreter could be useful.
-CHB
On Thu, Feb 20, 2020 at 8:51 AM Mike Miller <python-ideas@mgmiller.net> wrote:
On 2020-02-19 17:12, Bar Harel wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
As a heavy user of logging, I like this one a lot. In a new project, the lack of such a feature allows one to stick with print() perhaps longer than is prudent.
-Mike _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YTYC4X... Code of Conduct: http://python.org/psf/codeofconduct/
Let's plan a cmdline argument then.
Should we go for -L<level> much like -W<warning>?
Yeah, I think a command-line argument is much more reasonable, less likely to cause issues with other implementations of logging configuration, and more practically useful. I'd be +1 on something like a `-L<level>`. On Thu, Feb 20, 2020 at 2:05 PM Bar Harel <bzvi7919@gmail.com> wrote:
Let's plan a cmdline argument then.
Should we go for -L<level> much like -W<warning>? Or -x logging for setting a default value of debug or info?
Per python specification, -x is implementation dependent.
I'll advocate for -L. I think the same concerns as -W are relevant for all python implementations, and it gives more flexibility in terms of log level.
Next question: Should -x dev include it? If it was python day-1, I would say yes. My issue is the double inclusion of stream handlers in case the logging was bootstrapped already. So -x dev probably shouldn't include it.
On Thu, Feb 20, 2020, 8:28 PM Christopher Barker <pythonchb@gmail.com> wrote:
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system?
But a standard command line argument to the interpreter could be useful.
-CHB
On Thu, Feb 20, 2020 at 8:51 AM Mike Miller <python-ideas@mgmiller.net> wrote:
On 2020-02-19 17:12, Bar Harel wrote:
Another idea I've had that may be of use:
PYTHONLOGGING environment variable.
Setting PYTHONLOGGING to any log level or level name will initialize logging.basicConfig() with that appropriate level.
Another option would be that -x dev or a different -x logging will initialize basic config.
As a heavy user of logging, I like this one a lot. In a new project, the lack of such a feature allows one to stick with print() perhaps longer than is prudent.
-Mike _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/US4RHU... Code of Conduct: http://python.org/psf/codeofconduct/
-- Christopher Barker, PhD
Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YTYC4X... Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KGRSK4... Code of Conduct: http://python.org/psf/codeofconduct/
Christopher Barker wrote:
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system? But a standard command line argument to the interpreter could be useful.
Can you clarify what the concern is about other Python scripts running? Why doesn't that apply to [all the other special PYTHON* environment variables](https://docs.python.org/3/using/cmdline.html#environment-variables), such as `PYTHONWARNINGS` or `PYTHONVERBOSE`? Command line arguments only work if you're invoking the `python` command directly. If you're running something above that like a bash script or a package entrypoint it's more complicated. And configuring environment variables is sometimes easier than changing a command or editing a script, like when you're configuring a service hosted in the cloud (personally I'm thinking of AWS ECS). I say do the same thing as so many other options and offer both a command line argument and an environment variable. I don't think many people will be setting the environment variable in their global shell profiles. When I want to set PYTHONPATH I usually write: PYTHONPATH=/some/path python script.py which has no chance to affect anything else.
+1 On Thu, Feb 20, 2020 at 11:46 Alex Hall <alex.mojaki@gmail.com> wrote:
Christopher Barker wrote:
I think it’s a “Bad Idea” to use an environment variable — who knows what Python script may be running on a given system? But a standard command line argument to the interpreter could be useful.
Can you clarify what the concern is about other Python scripts running? Why doesn't that apply to [all the other special PYTHON* environment variables]( https://docs.python.org/3/using/cmdline.html#environment-variables), such as `PYTHONWARNINGS` or `PYTHONVERBOSE`?
Command line arguments only work if you're invoking the `python` command directly. If you're running something above that like a bash script or a package entrypoint it's more complicated. And configuring environment variables is sometimes easier than changing a command or editing a script, like when you're configuring a service hosted in the cloud (personally I'm thinking of AWS ECS).
I say do the same thing as so many other options and offer both a command line argument and an environment variable. I don't think many people will be setting the environment variable in their global shell profiles. When I want to set PYTHONPATH I usually write:
PYTHONPATH=/some/path python script.py
which has no chance to affect anything else. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LXLSYU... Code of Conduct: http://python.org/psf/codeofconduct/
-- --Guido (mobile)
Nice idea... in principle. But, what if my program has several loggers with several handlers each having its own logging info level? Would the Python's interpreter CLI argument (or the ENV variable) override all loggers and handlers logging levels? Why should CLI / ENV override those loggers and handlers? Ok... I know that the above-mentioned logging configuration is probably seldom used in real world... But logging module allows that kind of uses. For instance, in my case, I used to create a (root) logger with two handlers: a stream to stdout and log file. the stdout stream has a higher logging level (say, critical) than the log file (say, info) -- so most log records go to file while worst messages are also display in terminal. Additionally... if the logging level is changed by the program at some point, I think that the CLI / ENV logging level will become useless. Talking about me again, I used to create a logger with logging.getLogger and change immediately its logging level to the desired level with logger.setLevel. So, in my case, the CLI / ENV level would be overridden even before any log message was issued. Of course, most developers probably do not do that but... it can be done. Or not? Or are you suggesting a different approach? Are you suggesting that CLI / ENV would create an ex novo basicConfig logger (beyond program's loggers and handlers) and that all program's log messages would be sent to that CLI / ENV logger instead of (or also to) program's loggers? Thank you.
On 2020-02-21 08:45, jdveiga@gmail.com wrote:
Nice idea... in principle.
But, what if my program has several loggers with several handlers each having its own logging info level? Would the Python's interpreter CLI argument (or the ENV variable) override all loggers and handlers logging levels? Why should CLI / ENV override those loggers and handlers?
I believe that if you've already set up comprehensive logging, this technique is not needed. Sometimes such a set up can handle basicConfig() being called early, sometimes not. But it's optional, not default behavior. My understanding of the use case is one wants to see additional debug logs from project and dependencies without having to set up logging first. -Mike
-L<level> will act as logging.basicConfig(level=level), but on a temporary basis, much like -W sets warning's filters. If you don't setup logging, or don't want to (cause you're checking some library functions), this will be super handy. If you do setup logging, it has the same caveats as setting the warning system after you passed -W. (There will be a duplicate handler). Keep in mind that if you already had loggers with handlers set up, you won't need to use this flag, much like if you modified the default warnings you won't set -W. It's main use is for debugging purposes in the absent or in addition of logging configuration. On Fri, Feb 21, 2020, 7:51 PM Mike Miller <python-ideas@mgmiller.net> wrote:
On 2020-02-21 08:45, jdveiga@gmail.com wrote:
Nice idea... in principle.
But, what if my program has several loggers with several handlers each having its own logging info level? Would the Python's interpreter CLI argument (or the ENV variable) override all loggers and handlers logging levels? Why should CLI / ENV override those loggers and handlers?
I believe that if you've already set up comprehensive logging, this technique is not needed. Sometimes such a set up can handle basicConfig() being called early, sometimes not. But it's optional, not default behavior.
My understanding of the use case is one wants to see additional debug logs from project and dependencies without having to set up logging first.
-Mike _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BZZ5PO... Code of Conduct: http://python.org/psf/codeofconduct/
So, are you suggesting that -L must create a basic logger with the given logging level? Is it right? It can work...
Ok... Though some kind of logging configuration, even a default one, is always present. For example, when logging.info() module method --or some one similar-- is called for the first time, it creates a basic logger with a last resort handler if none exists --at least up to my knowledge. So automatic set up, but set up indeed. In any case... most important to me... I do not really know nothing about comprehensive logging. I made a search on web and on my books but does not find anything useful. Could you provide me any link or additional information on that matter, please? Thank you.
Last resort logger works only if no handlers are set. NullHandler for example is used on most libraries in order to prevent setting up the logging system, but it can cause errors not to show. Also, LastResort is set for the "Warning" level. -L<level> will give you the flexibility you need in order to debug issues. Theoretically you can solve it by -L ignoring null handlers and setting last resort handler to the given level, but that's too magicky. I believe using basicConfig is the cleanest and most user friendly solution. On Fri, Feb 21, 2020, 11:44 PM <jdveiga@gmail.com> wrote:
Ok... Though some kind of logging configuration, even a default one, is always present. For example, when logging.info() module method --or some one similar-- is called for the first time, it creates a basic logger with a last resort handler if none exists --at least up to my knowledge. So automatic set up, but set up indeed.
In any case... most important to me... I do not really know nothing about comprehensive logging. I made a search on web and on my books but does not find anything useful. Could you provide me any link or additional information on that matter, please?
Thank you. _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GLMJWT... Code of Conduct: http://python.org/psf/codeofconduct/
Did this idea ever went anywhere? I thought I saw a bpo issue for this but I can't find it anymore and it's not present in 3.8 or 3.9 NEWS nor in master as far as I can tell. I think it would be very useful, while it's possible to use argparse to have an argument to set it I've found that many novice users are afraid of logging and its various options, having to use argparse on top of it adds a layer of complexity so most of the time thay either just put print() everywhere, or hardcode the log level in their program making it complicated to change later. I usually start my scripts with: logging.basicConfig(level=os.environ.get('LOG_LEVEL', 'INFO')) but as everybody use a different environment variable or command line option it's not obvious how to set the log level for a given program without looking at its source code. Also I would like to add both an -L option and a PYTHONLOGGING environment variable. The command line option seems better when possible but when it's not possible to use it, either because the script uses a shebang, or because the command line arguments are not accessible (e.g. a lambda function on a cloud environment) being able to use the environment variable would be a nice alternative. Yes, it should not be set as a global variable environment but so should not PYTHONHASHSEED or PYTHONMALLOC. Sorry for the noise if this has already been implemented, if not I can start working on it. Rémi
Hey Remi, I've started working on it the other day, but unfortunately had difficulties with my development environment after suffering a hard drive failure. Haven't tried again since then. I've opened a quick bpo - 40886. Since my dev environment is still messed up and VStudio kills me with obscure error messages, you're welcome to give it a go :-) Thank you for being awesome! Bar Harel On Sat, Jun 6, 2020 at 1:40 AM <remi.lapeyre@henki.fr> wrote:
Did this idea ever went anywhere? I thought I saw a bpo issue for this but I can't find it anymore and it's not present in 3.8 or 3.9 NEWS nor in master as far as I can tell.
I think it would be very useful, while it's possible to use argparse to have an argument to set it I've found that many novice users are afraid of logging and its various options, having to use argparse on top of it adds a layer of complexity so most of the time thay either just put print() everywhere, or hardcode the log level in their program making it complicated to change later.
I usually start my scripts with:
logging.basicConfig(level=os.environ.get('LOG_LEVEL', 'INFO'))
but as everybody use a different environment variable or command line option it's not obvious how to set the log level for a given program without looking at its source code.
Also I would like to add both an -L option and a PYTHONLOGGING environment variable. The command line option seems better when possible but when it's not possible to use it, either because the script uses a shebang, or because the command line arguments are not accessible (e.g. a lambda function on a cloud environment) being able to use the environment variable would be a nice alternative. Yes, it should not be set as a global variable environment but so should not PYTHONHASHSEED or PYTHONMALLOC.
Sorry for the noise if this has already been implemented, if not I can start working on it.
Rémi _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BVUMUR... Code of Conduct: http://python.org/psf/codeofconduct/
I think the first thing that needs to be done, before creating a bpo issue, is to get consensus that it's both a plausible idea and a good idea, and I don't think that the discussion did either. Discussion starts here: https://mail.python.org/archives/list/python-ideas@python.org/thread/I74LVJW... Bar's initial reason for this: "Will be useful mostly for debugging purposes instead of temporarily modifying the code." Okay, but how does that work? I have a script: # myscript.py print("Hello world") and I want to add logging to it. So I set the environment variable, or use the proposed -L command line switch, and call the script: PYTHONLOGGING=mylogfile python myscript.py but what exactly gets logged? It's one thing to say that the interpreter takes care of importing logging and setting up a basic logger, and that's great, but I still have to import logging and call the `logging.info(...)` functions, so what does this proposal give us, *in detail* please, and how do I use it? I've read the entire thread and there is lot of detail there that seems to be written for expert users of logging. If you're an expert user of logging, you probably don't need an environment variable, or you can handle it yourself with os.environ.get('LOGLEVEL'). How is this going to impact basic and intermediate users of logging? The tutorial warns: "As [logging.basicConfig] is intended as a one-off simple configuration facility, only the first call will actually do anything: subsequent calls are effectively no-ops." So what happens if I have a script that explicitly calls basicConfig but I accidentally run it with the -L switch or PYTHONLOGGING? E.g. I may have an alias 'python=python -L', or I may have set the environment variable in a shell and then forgotten it. I sometimes have shells open for weeks at a time, I'm going to be terribly confused if myscript logs correctly in one open shell but not in another because of an environment variable I set two weeks earlier. -- Steven
On Sat, Jun 6, 2020 at 6:46 AM Steven D'Aprano <steve@pearwood.info> wrote:
I think the first thing that needs to be done, before creating a bpo issue, is to get consensus that it's both a plausible idea and a good idea, and I don't think that the discussion did either.
Discussion starts here:
https://mail.python.org/archives/list/python-ideas@python.org/thread/I74LVJW...
I was also under the impression that a new thread had been started, since that's what it looks like in my inbox, but everything is actually under that link already. Mailman is confusing.
Bar's initial reason for this:
"Will be useful mostly for debugging purposes instead of temporarily modifying the code."
Okay, but how does that work? I have a script:
# myscript.py print("Hello world")
and I want to add logging to it. So I set the environment variable, or use the proposed -L command line switch, and call the script:
PYTHONLOGGING=mylogfile python myscript.py
but what exactly gets logged?
I'm assuming you made a small mistake here, because `PYTHONLOGGING` would take a log level, not a filename.
It's one thing to say that the interpreter takes care of importing logging and setting up a basic logger, and that's great, but I still have to import logging and call the `logging.info(...)` functions, so what does this proposal give us, *in detail* please, and how do I use it?
My understanding is that it does something like: ``` import logging import os logging.basicConfig(level=os.environ["PYTHONLOGGING"]) ``` and that this is useful if you're using a library that uses logging, i.e. ` logging.info(...)` calls have already been written for you, you just want an easy way to activate them. I've read the entire thread and there is lot of detail there that seems
to be written for expert users of logging. If you're an expert user of logging, you probably don't need an environment variable, or you can handle it yourself with os.environ.get('LOGLEVEL').
How is this going to impact basic and intermediate users of logging?
I think for users familiar with environment variables, `PYTHONLOGGING=INFO python myscript.py` might be easier to remember than `import logging; logging.basicConfig(level='INFO')`. On the other hand, the Python code is fairly discoverable in an IDE. `import log` autocompletes `logging` for me, `logging.config` autocompletes `logging.basicConfig`, and from there it's easy to see the docstring and parameters. And adding more ways of doing things can make learning harder.
The tutorial warns: "As [logging.basicConfig] is intended as a one-off simple configuration facility, only the first call will actually do anything: subsequent calls are effectively no-ops."
So what happens if I have a script that explicitly calls basicConfig but I accidentally run it with the -L switch or PYTHONLOGGING?
E.g. I may have an alias 'python=python -L', or I may have set the environment variable in a shell and then forgotten it. I sometimes have shells open for weeks at a time, I'm going to be terribly confused if myscript logs correctly in one open shell but not in another because of an environment variable I set two weeks earlier.
I think this last point is interesting, and puts me in favour of explicit calls to basicConfig overriding the environment variable, so the environment variable wouldn't be *exactly* equivalent to a call to basicConfig.
It's one thing to say that the interpreter takes care of importing logging and setting up a basic logger, and that's great, but I still have to import logging and call the `logging.info(...)` functions, so what does this proposal give us, *in detail* please, and how do I use it? You would be able to easily see the logs already generated by libraries, without needing to ever modify the Python files. It's much like -Xdev, -Wdefault etc. How is this going to impact basic and intermediate users of logging? Without the need of entering logging internals, you would be able to see the logs, or be able to easily create a debug environment with the env var, regardless of the script running underneath. So what happens if I have a script that explicitly calls basicConfig but I accidentally run it with the -L switch or PYTHONLOGGING? It's a good question. Unlike Alex's suggestion, my original point of view is that it will be exactly like basicConfig. If someone wrote basicConfig(level=INFO), and you wish to see everything regardless of current setup (mostly for pinpointing bugs), you'd be able to set it to DEBUG in an easy way, that will work on all scripts. E.g. I may have an alias 'python=python -L', or I may have set the environment variable in a shell and then forgotten it. I sometimes have shells open for weeks at a time, I'm going to be terribly confused if myscript logs correctly in one open shell but not in another because of an environment variable I set two weeks earlier. Well, PYTHONPATH, PYTHONWARNINGS and all other env vars will cause the same issue and confusion. I think it's up to the developer to know how to set up his own environment. Keep in mind that all arguments in favor of this, and all arguments against, also apply to PYTHONWARNINGS. It's exactly there for when you wish to show suppressed deprecation warnings on dev environments, or further customize the filter. Adding this will offer a more consistent API, as there's no reason to make an actual distinction between the two. But that's just icing on the cake IMO. I can give you an elaboration of a few hard-to-debug cases that made me surprised such a feature does not exist. Bar Hare On Sat, Jun 6, 2020, 7:46 AM Steven D'Aprano <steve@pearwood.info> wrote:
I think the first thing that needs to be done, before creating a bpo issue, is to get consensus that it's both a plausible idea and a good idea, and I don't think that the discussion did either.
Discussion starts here:
https://mail.python.org/archives/list/python-ideas@python.org/thread/I74LVJW...
Bar's initial reason for this:
"Will be useful mostly for debugging purposes instead of temporarily modifying the code."
Okay, but how does that work? I have a script:
# myscript.py print("Hello world")
and I want to add logging to it. So I set the environment variable, or use the proposed -L command line switch, and call the script:
PYTHONLOGGING=mylogfile python myscript.py
but what exactly gets logged?
It's one thing to say that the interpreter takes care of importing logging and setting up a basic logger, and that's great, but I still have to import logging and call the `logging.info(...)` functions, so what does this proposal give us, *in detail* please, and how do I use it?
I've read the entire thread and there is lot of detail there that seems to be written for expert users of logging. If you're an expert user of logging, you probably don't need an environment variable, or you can handle it yourself with os.environ.get('LOGLEVEL').
How is this going to impact basic and intermediate users of logging?
The tutorial warns: "As [logging.basicConfig] is intended as a one-off simple configuration facility, only the first call will actually do anything: subsequent calls are effectively no-ops."
So what happens if I have a script that explicitly calls basicConfig but I accidentally run it with the -L switch or PYTHONLOGGING?
E.g. I may have an alias 'python=python -L', or I may have set the environment variable in a shell and then forgotten it. I sometimes have shells open for weeks at a time, I'm going to be terribly confused if myscript logs correctly in one open shell but not in another because of an environment variable I set two weeks earlier.
-- Steven _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TQM7ZA... Code of Conduct: http://python.org/psf/codeofconduct/
participants (9)
-
Alex Hall -
Bar Harel -
Christopher Barker -
Guido van Rossum -
jdveiga＠gmail.com -
Kyle Stanley -
Mike Miller -
remi.lapeyre＠henki.fr -
Steven D'Aprano