[issue41449] An article on Python 3 stdout and stderr output buffering
New submission from Anatoli Babenia <anatoli@rainforce.org>: It is hard to find info why Python 3 buffers stdout/stderr. The buffering causes problems when debugging Python apps in Docker and Kubernetes, and it is unclear if it is Python 3 who starts to buffer stdout if no tty is attached, it is Docker, or it is Kubernetes. The only bit of info that could be searched is the description of -u option https://docs.python.org/3.8/using/cmdline.html?#cmdoption-u which is not linked to any article. The `-u` description also says.
Changed in version 3.7: The text layer of the stdout and stderr streams now is unbuffered.
However, I don't understand what is the text layers of stdout. And there is no description of behaviour when the output is not attached, and when the output is redirected. ---------- assignee: docs@python components: Documentation messages: 374633 nosy: Anatoli Babenia, docs@python priority: normal severity: normal status: open title: An article on Python 3 stdout and stderr output buffering _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Change by tedder <ted@perljam.net>: ---------- nosy: +tedder _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Guido van Rossum <guido@python.org> added the comment: Maybe you’ll get some explanation or pointers on python-list. Or were you volunteering to write something? ---------- nosy: +gvanrossum _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Anatoli Babenia <anatoli@rainforce.org> added the comment: Not sure I can volunteer - need to find a sustenance ASAP. But given that nothing moved in this direction over the past 6 months, another week spent not writing this stuff won't change anything. In any case it would start with the following problem - "Logs don't show up if running in Kubernetes" - https://github.com/bottlepy/bottle/issues/1130 I've encountered it a 3 days session on adding CI to my favourite cheat sheet server https://github.com/chubin/cheat.sh/pull/224/commits/10bfaab38cc649d765637bf4... and I am still not sure I am right. That's why I want to read something about it. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Guido van Rossum <guido@python.org> added the comment: I don't think that waiting another six months is going to make this happen either. Maybe you can petition realpython.com to publish something about this. Honestly it sounds like this ties too many different topics together to easily fit in the library docs: in order to make a large number of different use cases easy, how I/O works under the hood in Python is truly complex. So I'm going to close this rather than have it add to our issue count. ---------- resolution: -> wont fix stage: -> resolved status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Anatoli Babenia <anatoli@rainforce.org> added the comment: To avoid too much outside reference, the commit message I am unsure of is this.
Run Python 3 in unbuffered mode to see lines in `docker logs` as soon as they appear. When Python 3 is not attached to a terminal, it turns on the buffering, like when `docker` runs in a background.
I am not sure. 1. How Python detects it is attached to terminal or not. 2. That it is Python that turns on stdout buffering. 3. What happens to stdout with child processes when they run with docker in different modes - foreground, background (-d), interactive (-i), with tty (-t) and without tty. ---------- resolution: wont fix -> status: closed -> open _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Guido van Rossum <guido@python.org> added the comment: I don't know anything about Docker, so I can't help you there. Please don't reopen the issue again. ---------- resolution: -> wont fix status: open -> closed _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Anatoli Babenia <anatoli@rainforce.org> added the comment: I didn't reopen the issue explicitly. Just pressed the submit button again when the notification about edited bug appeared. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Anatoli Babenia <anatoli@rainforce.org> added the comment: The point is that without a clear description of buffering in Python 3, it is impossible to understand what goes when Python is wrapped by Docker, supervisord, redirected to file etc. ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
Guido van Rossum <guido@python.org> added the comment: If you don't even know whether it's Python, Docker or Kubernetes you really need to ask somewhere else. Plenty of user groups around. Python detects tty using the standard UNIX isatty() function. To a tty we always get line buffering. To a file we use a larger buffer. For more details see the source at https://github.com/python/cpython/blob/4660597b51b3d14ce6269d0ed865ab7e22c6a... ---------- _______________________________________ Python tracker <report@bugs.python.org> <https://bugs.python.org/issue41449> _______________________________________
participants (3)
-
Anatoli Babenia -
Guido van Rossum
-
tedder