What should go to stdout/stderr and why Python logging write everything to stderr?

[Cameron Simpson]

On Jan 3, 2023, at 10:38 AM, c.buhtz at posteo.jp wrote:
>> this posting isn't about asking for a technical solution. My 
>> intention is to understand the design decision Python's core developers made in
>> context of that topic.
>>
>> The logging module write everything to stderr no matter which logging
>> level is used.

Sure. They're logs.

>> The argparse module does it more differentiated. If arguments are
>> mandatory but none are given then argparse produce a short version of
>> the usage info; on stderr. If the user request the usage info via "-h"
>> it goes to stdout. This is the behavior I would expect.

Error messages to stderr. _Requested output_ to stdout. Good with me.

>> Why does logging behave different? DEBUG and INFO imho should go to
>> stdout not stderr.

I entirely disagree.

Consider a shell pipeline:

     cmd1 | cmd2 | cmd3

where `cmd2` and `cmd3` parse data coming down the pipes. Letting 
logging info into those data streams would produce garbage. This 
separation between core programme output and error/logging output is the 
basic reason that the two streams exist, and the reason logging goes to 
stderr by default.

You're conflating "normal output" with INFO, which is not a substitute 
for eg `print()`.

>> Of course I could modify the handlers etc. to workaround this. But I
>> assume that there was something in mind of the Python developers when
>> they decided that.

Basicly, logging data is not the normal output. So it does not get sent 
to stdout. The log level doesn't change that characteristic.

>> My goal is not to divide between the use of print() or logging.info()
>> in my own code. This would mess up a lot.

I think you should, or you should fiddle the handlers as you previous 
resist doing if it is your goal.

But keep in mind that if you do that globally (eg in the root logger) it 
will affect _all_ logging calls, not merely your own, and other 
libraries will assume they can log at whatever level and not pollute 
stdout with their logs.

Basicly, logging isn't "output".

Cheers,
Cameron Simpson <cs at cskk.id.au>