<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Jan 11, 2010, at 1:47 PM Nobody <<a href="mailto:nobody@nowhere.com">nobody@nowhere.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; "><font class="Apple-style-span" color="#0000007D"><b><br></b></font></div><blockquote type="cite">On Mon, 11 Jan 2010 10:09:36 +0100, Martin v. Loewis wrote:<br><br></blockquote><blockquote type="cite"><blockquote type="cite">In Python 3.1 is there any difference in the buffering behavior of the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">initial sys.stdout and sys.stderr streams?<br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">No.<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite"><blockquote type="cite">Were they different at some earlier point in Python's evolution?<br></blockquote></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">That depends on the operating system. These used to be whatever the<br></blockquote><blockquote type="cite">C library set up as stdout and stderr. Typically, they were buffered<br></blockquote><blockquote type="cite">in the same way.<br></blockquote><br>On Unix, stdout will be line buffered if it is associated with a tty<span class="Apple-converted-space"> </span><br>and fully buffered otherwise, while stderr is always unbuffered.<br><br>On Windows, stdout and stderr are unbuffered if they refer to a character<br>device, fully buffered otherwise (Windows doesn't have line buffering;<br>setvbuf(_IOLBF) is equivalent to setvbuf(_IOFBF)).<br><br>ANSI C says:<br><br>As initially opened, the standard error stream is not fully buffered; the<br>standard input and standard output streams are fully buffered if and only<br>if the stream can be determined not to refer to an interactive device.<span class="Apple-converted-space"> </span><br><br></span></blockquote></div><br><div>I don't want to get into a quibble fight here, but I need to reraise this issue.</div><div>[I teach and write and want to make sure I get this right. I already have an</div><div>incorrect paragraph about this in my Bioinformatics Programming Using Python</div><div>book.] The key question here is line buffering vs full buffering. In Unix (at least</div><div>in an OS X Terminal), the following code prints a number every two seconds</div><div>in Python 2:</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>>>> for n in range(5):</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>. . . <span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>print >> sys.stderr, n, # final , to not send newline</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>. . .<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>time.sleep(2)</div><div><br></div><div>However, in Python 3, similar code does not print the numbers until the whole</div><div>thing finishes (again, running from the terminal).</div><div><br></div><div><div><span class="Apple-tab-span" style="white-space: pre; "> </span>>>> for n in range(5):</div><div><span class="Apple-tab-span" style="white-space: pre; "> </span>. . . <span class="Apple-tab-span" style="white-space: pre; "> </span><span class="Apple-tab-span" style="white-space: pre; "> </span>print(n, file=sys.stderr, end='')</div><div><span class="Apple-tab-span" style="white-space: pre; "> </span>. . .<span class="Apple-tab-span" style="white-space: pre; "> </span><span class="Apple-tab-span" style="white-space: pre; "> </span>time.sleep(2)</div><div><br></div><div>So it appears that in a Unix terminal window, Python 2 does not line-buffer stderr</div><div>whereas Python 3 does. That's what tripped me up. While developing and debugging</div><div>code, I often print periods on a line as some loop progresses (sometimes every Nth</div><div>time around, for some reasonable N) just to know the pace of execution and that</div><div>the program is still doing something. In doing that recently in Python 3 I discovered that</div><div>I either had to leave out the end='' or do sys.stderr.flush() after every print, which</div><div>amounts to the same thing.</div><div><br></div><div>This was a big surprise, after many, many years of C, C++,</div><div>Java, and Python programming -- I have always thought of stderr as completely</div><div>unbuffered in languages that have it. Doesn't mean some languages line-buffer</div><div>stderr on some platforms, just pointing out an assumption I've lived with for a very</div><div>long time that tripped me up writing a note about using stderr in Python 3 without</div><div>actually demonstrating the code and therefore not catching my error.</div></div></body></html>