[Python-Dev] Loggers in the stdlib and logging configuration APIs

[Vinay Sajip]

Glenn Linderman <v+python <at> g.nevcal.com> writes:

>     I thought of the idea of a flag to make loggers immune, but it
>     seemed it could lead to an arms race, where the knee-jerk reaction
>     of discovering that a library's logger got disabled would be to set
>     the flag, and, of course, the knee-jerk reaction of discovering that
>     a library's logger produced an unwanted message would cause the
>     application to use the new option, to "turn them all off again". 
>     Then a new library logger option, to become immune to the new
>     disable them all option, would be requested.  Etc.

Although an arms race is theoretically possible, I doubt it would come to
that. After all, people can just set a logger's disabled flag to False, and
its messages would start flowing again. That would be as easy for users as
making any new API call.

>     That's why I suggested a list, although perhaps your scheme would be
>     "more" backward compatible, as it defines a meaning for the default
>     behavior that sounds somewhat similar to the current default
>     behavior.... but isn't, really.
>     Interesting that you choose to disable all loggers not mentioned
>     explicitly (a logger white list), whereas I suggested listing the
>     ones to disable (a logger black list).

That's the backward compatible behaviour. In general, when the configuration
APIs are called, the design assumption was that you want to replace any
existing configuration with a new one;  that now needs to change because of
places where logging will be used for notifications where exceptions cannot be
usefully raised.

>     I suppose you suggested the white list, figuring that the
>     application could choose which messages it wishes to see.

An application can always choose not to see or not see messages via setting
verbosity levels on specific loggers and/or handlers to the desired values.

>     I chose a black list, figuring that new messages being introduced
>     might be useful to the application, but if they never see them
>     (because they are not in the white list), they wouldn't know about
>     them.  Of course, either scheme can be programmed around, given an

Don't forget that any loggers created after a fileConfig/dictConfig call
wouldn't be disabled; and for many many uses, logging will be set up
reasonably early in the application's execution, and the only loggers which
would be disabled by a configuration call would be loggers created before the
call, and which are not explicitly named in the configuration. There will be
relatively few of those.

>     messages.  Applications that use logging, really should be prepared
>     for new messages to be displayed.  Testing won't discover them all,
>     disabling them could result in useful messages being suppressed. 

Yes, because third party libraries can introduce new messages either by
themseleves or through adding a new dependency, but disabling via a config
call is unlikely if the config call is made early.
 
>     Should a module's documentation be expected to include lists of log
>     messages that it may produce at any level of logging?

That would never be kept up to date :-) The best we can hope for is for module
authors to document that they use logging, mention any logger names used, the
default level settings for at least the top-level logger, and any convenience
APIs provided for configuring handlers.

Suppose we just implement a "leave_enabled" flag on loggers and change the
config code to honour this. Then, there will be no backward incompatible
change until we modify stdlib loggers to set leave_enabled to True (I'm not
sure if this could be done for concurrent.futures now as it's a new package in
3.2; it depends on whether this would be regarded as a new feature).

I don't propose adding another optional argument to the configuration APIs
unless and until there seems to be a need for it; in the absence of this
argument, users would have to  explicitly turn off loggers they wanted to
silence in the normal way (i.e. by setting levels, and perhaps propagation
flags).

I would propose to document "leave_enabled" as part of the public API, but
with a recommendation that due care and consideration be given before choosing
to set it for a logger.

How does that sound?

Regards,


Vinay Sajip