Trouble propagating logging configuration
Loris Bennett
loris.bennett at fu-berlin.de
Thu Sep 2 03:06:53 EDT 2021
"Dieter Maurer" <dieter at handshake.de> writes:
> Loris Bennett wrote at 2021-9-1 13:48 +0200:
>> ...
>>Yes, but to quote from https://docs.python.org/3.6/howto/logging.html#logging-basic-tutorial:
>>
>> A good convention to use when naming loggers is to use a module-level
>> logger, in each module which uses logging, named as follows:
>>
>> logger = logging.getLogger(__name__)
>>
>> This means that logger names track the package/module hierarchy, and
>> it’s intuitively obvious where events are logged just from the logger
>> name.
>>
>>so in this case the source layout is relevant, isn't it?
>
> Relevant in this case is the package/module hierarchy.
>
> Often the package/module hierarchy follows the source layout
> **BUT** this is not necessarily the case.
>
> In particular, the "start" module of a script is called `__main__`
> indepently of its location.
> Furthermore, so called "namespace packages" consist of
> decentralized (i.e. located at different places in the file system)
> subpackages.
>
> Thus, in general, the connection between pachage/module hierarchy
> and the source layout is loose.
>
>
>>> Furthermore, the place of the configuration (and where in the
>>> code it is activated) is completely irrelevant for the "inheritance".
>>
>>OK, so one issue is that I was getting confused by the *order* in which
>>modules are being called. If I have two modules, 'foo' and 'bar', in
>>the same directory, configure the logging just in 'foo' and then call
>>
>>
>> foo.some_method()
>> bar.some_method()
>>
>>then both methods will be logged. If I do
>>
>> bar.some_method()
>> foo.some_method()
>>
>>then only the method in 'foo' will be logged.
>
> Usually, log configuration is considered a (global) application
> (not a (local) package/module) concern:
> The components (modules) decide what to log at what level
> and global configuration decides what to do with those messages.
>
> Thus, typically, you configure the complete logging in
> your main application module and then start to import modules
> and call their functions.
>
>> ...
>>If I have
>>
>> [loggers]
>> keys=root,main,log_test
>>
>>in my logging configuration and initialise the logging in run.py ...
>
> This logging configuration is obviously not complete (thus, I cannot
> check what happens in your case).
> You may have made errors at other places in your configuration
> which may explain your observations.
>
> At your place, I would look at the `logging` source code
> to find out where you can see (in an interactive Python session)
> which loggers have been created by your configuration and
> how they are configured. For the last part you use "vars(logger)".
Thanks Peter and Dieter for all the help. I have finally figured out
what my problem was. If in a module 'mylibs.mylib' I have
import logging
logger = logging.getLogger(__name__)
and in my main script have
import logger
import logger.config
import mylibs.mylib
logging.config.fileConfig("/home/loris/config")
logger = logging.getLogger(__name__)
then 'logger' in 'mylibs.mylib' is defined before
'logging.config.fileConfig' is called and the logger in the module is
not configured.
If change the order and write
import logger
import logger.config
logging.config.fileConfig("/home/loris/config")
logger = logging.getLogger(__name__)
import mylibs.mylib
then the 'logger' in 'mylibs.mylibs' does get configured properly.
I'm still thinking what implications this has if I want to load a
configuration file via a command-line option.
Cheers,
Loris
--
This signature is currently under construction.
More information about the Python-list
mailing list