What should go to stdout/stderr and why Python logging write everything to stderr?

[Chris Angelico]

On Wed, 4 Jan 2023 at 16:21, Grant Edwards <grant.b.edwards at gmail.com> wrote:
>
> On 2023-01-04, Chris Angelico <rosuav at gmail.com> wrote:
> > On Wed, 4 Jan 2023 at 09:52, Grant Edwards <grant.b.edwards at gmail.com> wrote:
> >>
> >>> I can't think of a specific example, but I know I have piped the output
> >>> of a program while at the same time interacting with a prompt on stderr.
> >>> A rare thing, though.
> >>
> >> Programs that ask for passwords often do it by reading/writing to
> >> fd 0 or /dev/tty so that stdout is unaffected.
> >
> > Reading/writing the FD is the same as using stdout.
>
> No, stdout is fd 1.
>
> > (technically you'd write to fd 1 and read from fd 0),

I assumed that you were using a shorthand. My bad.

> No, the whole point is _not_ to write to stdout (which would corrupt
> the program's output). The way I remember it was that you wrote the
> prompt to fd 0, and then read the password from fd 0. That way it
> worked even when fd 1 (stdout) was redirected. It still failed if
> stdin was redirected...

Writing to FD 0 assumes that (a) it was originally connected to the
TTY, and (b) it hasn't been redirected. Not a reliable assumption.

rosuav at sikorsky:~/tmp$ cat testfds.py
import os
for fd in range(3):
    os.write(fd, b"This is on FD %d\n" % fd)
rosuav at sikorsky:~/tmp$ python3 testfds.py
This is on FD 0
This is on FD 1
This is on FD 2
rosuav at sikorsky:~/tmp$ echo | python3 testfds.py
Traceback (most recent call last):
  File "/home/rosuav/tmp/testfds.py", line 3, in <module>
    os.write(fd, b"This is on FD %d\n" % fd)
OSError: [Errno 9] Bad file descriptor

Writing to fd 0 might work if stdout is redirected, but it won't work
if stdin is redirected, so honestly, I don't think this hack is at all
reliable.

> > but yes, can use /dev/tty to reach for the console directly.
>
> That's definitely a better option, but I'm pretty sure I've seen
> multiple examples over the decades where fd 0 was used instead. Was
> /dev/tty a "recent" enough invention that I would have seen
> application code that was written before it existed? Or maybe I was
> just looking at code written by people who weren't aware of /dev/tty?

No idea. Probably. It's definitely possible that someone tries it with
stdout redirected and goes "oh hey, writing to fd 0 still works", and
never tries it with stdin redirected. It's very common for programmers
to see that something works without knowing WHY it works :)

By the way, I can reinstate the behaviour of "FD 0 writes to the TTY"
in bash by redirecting it:

rosuav at sikorsky:~/tmp$ python3 testfds.py >/dev/null 0<>/dev/tty
This is on FD 0
This is on FD 2

which is another neat demonstration that bash has more power than you
will ever need...

(though redirecting FDs above 2 in read/write mode can be *incredibly* useful.)

ChrisA