Replacing the standard IO streams (was Re: changing sys.stdout encoding)
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following: sys.stdin = open(sys.stdin.fileno(), 'r', <new settings>) sys.stdout = open(sys.stdout.fileno(), 'w', <new settings>) sys.stderr = open(sys.stderr.fileno(), 'w', <new settings>) Ditto for the other standard streams. It seems it already *is* as simple as with any other file, we just collectively forgot about: 1. The fact open() accepts file descriptors directly in Python 3 2. The fact that text streams still report the underlying file descriptor correctly *That* is something we can happily advertise in the standard library docs. If you could check to make sure it works properly for your use case and then file a docs bug at bugs.python.org to get it added to the std streams documentation, that would be very helpful. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 9 June 2012 10:55, Nick Coghlan <ncoghlan@gmail.com> wrote:
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following:
sys.stdin = open(sys.stdin.fileno(), 'r', <new settings>) sys.stdout = open(sys.stdout.fileno(), 'w', <new settings>) sys.stderr = open(sys.stderr.fileno(), 'w', <new settings>)
Ditto for the other standard streams. It seems it already *is* as simple as with any other file, we just collectively forgot about:
One minor point - if sys.stdout is redirected, *and* you have already written to sys.stdout, this resets the file pointer. With test.py as import sys print("Hello!") sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") test.py >a gives one line in a, not two (tested on Windows, Unix may be different). And changing to "a" doesn't resolve this... Of course, the actual use case is to change the encoding before anything is written - so maybe a small note saying "don't do this" is enough. But it's worth mentioning before we get the bug report saying "Python lost my data" :-) Paul.
On 9 June 2012 12:00, Paul Moore <p.f.moore@gmail.com> wrote:
On 9 June 2012 10:55, Nick Coghlan <ncoghlan@gmail.com> wrote:
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following:
sys.stdin = open(sys.stdin.fileno(), 'r', <new settings>) sys.stdout = open(sys.stdout.fileno(), 'w', <new settings>) sys.stderr = open(sys.stderr.fileno(), 'w', <new settings>)
Ditto for the other standard streams. It seems it already *is* as simple as with any other file, we just collectively forgot about:
One minor point - if sys.stdout is redirected, *and* you have already written to sys.stdout, this resets the file pointer. With test.py as
import sys print("Hello!") sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!")
test.py >a gives one line in a, not two (tested on Windows, Unix may be different). And changing to "a" doesn't resolve this...
Ignore me - you need to flush stdout before repoening it, is all. Dumb mistake, sorry for the noise :-( Paul.
On 09/06/2012 12:00, Paul Moore wrote:
On 9 June 2012 10:55, Nick Coghlan<ncoghlan@gmail.com> wrote:
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following:
sys.stdin = open(sys.stdin.fileno(), 'r',<new settings>) sys.stdout = open(sys.stdout.fileno(), 'w',<new settings>) sys.stderr = open(sys.stderr.fileno(), 'w',<new settings>)
Ditto for the other standard streams. It seems it already *is* as simple as with any other file, we just collectively forgot about:
One minor point - if sys.stdout is redirected, *and* you have already written to sys.stdout, this resets the file pointer. With test.py as
import sys print("Hello!") sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!")
test.py>a gives one line in a, not two (tested on Windows, Unix may be different). And changing to "a" doesn't resolve this...
Of course, the actual use case is to change the encoding before anything is written - so maybe a small note saying "don't do this" is enough. But it's worth mentioning before we get the bug report saying "Python lost my data" :-)
I find that this: print("Hello!") sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") prints the string "Hello!\r\r\n", but this: print("Hello!") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") prints the string "Hello!\r\nHello!\r\r\n". I had hoped that the flush would be enough, but apparently not.
On 09.06.12 12:55, Nick Coghlan wrote:
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following:
sys.stdin = open(sys.stdin.fileno(), 'r',<new settings>) sys.stdout = open(sys.stdout.fileno(), 'w',<new settings>) sys.stderr = open(sys.stderr.fileno(), 'w',<new settings>)
sys.stdin = io.TextIOWrapper(sys.stdin.detach(), <new settings>) sys.stdout = io.TextIOWrapper(sys.stdout.detach(), <new settings>) ... None of these methods are not guaranteed to work if the input or output have occurred before.
On 09/06/2012 21:02, Serhiy Storchaka wrote:
None of these methods are not guaranteed to work if the input or output have occurred before.
That's a double negative so I'm not sure what you meant to say. Can you please rephrase it. I assume that English is not your native language, so I'll let you off :) -- Cheers. Mark Lawrence.
On 10.06.12 00:22, Mark Lawrence wrote:
On 09/06/2012 21:02, Serhiy Storchaka wrote:
None of these methods are not guaranteed to work if the input or output have occurred before.
That's a double negative so I'm not sure what you meant to say. Can you please rephrase it. I assume that English is not your native language, so I'll let you off :)
open(sys.stdin.fileno()) is not guaranteed to work if the input or output have occurred before. And io.TextIOWrapper(sys.stdin.detach()) is not guaranteed to work if the input or output have occurred before. sys.stdin internal buffer can contains read by not used characters. sys.stdin.buffer internal buffer can contains read by not used bytes. With multibyte encoding sys.stdin.decoder internal buffer can contains uncompleted multibyte character.
On Mon, Jun 11, 2012 at 12:34 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
On 10.06.12 00:22, Mark Lawrence wrote:
On 09/06/2012 21:02, Serhiy Storchaka wrote:
None of these methods are not guaranteed to work if the input or output have occurred before.
That's a double negative so I'm not sure what you meant to say. Can you please rephrase it. I assume that English is not your native language, so I'll let you off :)
open(sys.stdin.fileno()) is not guaranteed to work if the input or output have occurred before. And io.TextIOWrapper(sys.stdin.detach()) is not guaranteed to work if the input or output have occurred before. sys.stdin internal buffer can contains read by not used characters. sys.stdin.buffer internal buffer can contains read by not used bytes. With multibyte encoding sys.stdin.decoder internal buffer can contains uncompleted multibyte character.
Right, but the point of this discussion is to document the cleanest available way for an application to change these settings at *application start* (e.g. to support an "--encoding" parameter). Yes, there are potential issues if you use any of these mechanisms while there is data in the buffers, but that's a much harder problem and not one we're trying to solve here. Regardless, the advantage of the "open + fileno" idiom is that it works for *any* level of change. If you want to force your streams to unbuffered binary IO rather than merely changing the encoding: sys.stdin = open(sys.stdin.fileno(), 'rb', buffering=0, closefd=False) sys.stdout = open(sys.stdout.fileno(), 'wb', buffering=0, closefd=False) sys.stderr = open(sys.stderr.fileno(), 'wb', buffering=0, closefd=False) Keep them as text, but force them to permissive utf-8, no matter how the interpreter originally created them?: sys.stdin = open(sys.stdin.fileno(), 'r', encoding="utf-8", errors="surrogateescape", closefd=False) sys.stdout = open(sys.stdout.fileno(), 'w', encoding="utf-8", errors="surrogateescape", closefd=False) sys.stderr = open(sys.stderr.fileno(), 'w', encoding="utf-8", errors="surrogateescape", closefd=False) This approach also has the advantage of leaving sys.__std(in/out/err)__ in a somewhat usable state. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
^^^^^^^[[[[[[[[[@[@Nick Coghlan writes:
On Mon, Jun 11, 2012 at 12:34 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
open(sys.stdin.fileno()) is not guaranteed to work if the input or output have occurred before. [...]
Right, but the point of this discussion is to document the cleanest available way for an application to change these settings at *application start* (e.g. to support an "--encoding" parameter). Yes, there are potential issues if you use any of these mechanisms while there is data in the buffers,
+1 The OP's problem is a real one. His use case (the "--encoding" parameter) seems to be the most likely one in production use, so the loss of buffered data issue should rarely come up. Changing encodings on the fly offers plenty of ways to lose data besides incomplete buffers, anyway. I am a little concerned with MRAB's report that import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello") doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
^^^^^^^[[[[[[[[[@[@Nick Coghlan writes:
On Mon, Jun 11, 2012 at 12:34 AM, Serhiy Storchaka<storchaka@gmail.com> wrote:
open(sys.stdin.fileno()) is not guaranteed to work if the input or output have occurred before. [...]
Right, but the point of this discussion is to document the cleanest available way for an application to change these settings at *application start* (e.g. to support an "--encoding" parameter). Yes, there are potential issues if you use any of these mechanisms while there is data in the buffers,
+1
The OP's problem is a real one. His use case (the "--encoding" parameter) seems to be the most likely one in production use, so the loss of buffered data issue should rarely come up. Changing encodings on the fly offers plenty of ways to lose data besides incomplete buffers, anyway.
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right: Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n" All on Windows.
On 10 June 2012 19:12, MRAB <python@mrabarnett.plus.com> wrote:
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
Not here (Win 7 32-bit): PS D:\Data> type t.py import sys print("Hello!") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") PS D:\Data> py -3.2 t.py | od -c 0000000 H e l l o ! \r \n H e l l o ! \r \n 0000020 Paul.
On 10/06/2012 19:34, Paul Moore wrote:
On 10 June 2012 19:12, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
Not here (Win 7 32-bit):
PS D:\Data> type t.py import sys print("Hello!") sys.stdout.flush()
sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") PS D:\Data> py -3.2 t.py | od -c 0000000 H e l l o ! \r \n H e l l o ! \r \n 0000020
I'm using Windows XP Pro (32-bit), initially sys.stdout.encoding == "cp1252".
On 10 June 2012 20:01, MRAB <python@mrabarnett.plus.com> wrote:
On 10/06/2012 19:34, Paul Moore wrote:
On 10 June 2012 19:12, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
Not here (Win 7 32-bit):
PS D:\Data> type t.py import sys print("Hello!") sys.stdout.flush()
sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") PS D:\Data> py -3.2 t.py | od -c 0000000 H e l l o ! \r \n H e l l o ! \r \n 0000020
I'm using Windows XP Pro (32-bit), initially sys.stdout.encoding == "cp1252".
PS D:\Data> py -3 -c "import sys; print(sys.stdout.encoding)" cp850 This is at the console (Powershell) - are you running from within something like idle, or a GUI environment? Paul.
On 10/06/2012 21:07, Paul Moore wrote:
On 10 June 2012 20:01, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 19:34, Paul Moore wrote:
On 10 June 2012 19:12, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
Not here (Win 7 32-bit):
PS D:\Data> type t.py import sys print("Hello!") sys.stdout.flush()
sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") PS D:\Data> py -3.2 t.py | od -c 0000000 H e l l o ! \r \n H e l l o ! \r \n 0000020
I'm using Windows XP Pro (32-bit), initially sys.stdout.encoding == "cp1252".
PS D:\Data> py -3 -c "import sys; print(sys.stdout.encoding)" cp850
This is at the console (Powershell) - are you running from within something like idle, or a GUI environment?
It's at the system command prompt. When I redirect the script's stdout to a file (on the command line using ">output.txt") I get those 15 bytes from Python 3.2. Your output appears to be 32 bytes (the second line starts with "0000020").
On 10 June 2012 21:28, MRAB <python@mrabarnett.plus.com> wrote:
On 10/06/2012 21:07, Paul Moore wrote:
On 10 June 2012 20:01, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 19:34, Paul Moore wrote:
On 10 June 2012 19:12, MRAB<python@mrabarnett.plus.com> wrote:
On 10/06/2012 17:41, Stephen J. Turnbull wrote:
I am a little concerned with MRAB's report that
import sys print("hello") sys.stdout.flush() sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("hello")
doesn't work as expected, though. (It does work for me on Mac OS X, both as above -- of course there are no '\r's in the output -- and with 'print("hello", end="\r\n")'.)
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
Not here (Win 7 32-bit):
PS D:\Data> type t.py import sys print("Hello!") sys.stdout.flush()
sys.stdout = open(sys.stdout.fileno(), 'w', encoding='utf-8') print("Hello!") PS D:\Data> py -3.2 t.py | od -c 0000000 H e l l o ! \r \n H e l l o ! \r \n 0000020
I'm using Windows XP Pro (32-bit), initially sys.stdout.encoding == "cp1252".
PS D:\Data> py -3 -c "import sys; print(sys.stdout.encoding)" cp850
This is at the console (Powershell) - are you running from within something like idle, or a GUI environment?
It's at the system command prompt. When I redirect the script's stdout to a file (on the command line using ">output.txt") I get those 15 bytes from Python 3.2.
Your output appears to be 32 bytes (the second line starts with "0000020").
Well spotted - PowerShell does funny things with Unicode in pipes, I'd forgotten. Indeed, I get the same output as you from cmd. Odd. Paul
MRAB writes:
That's actually Python 3.1. From Python 3.2 it's slightly different, but still not quite right:
Python 3.1: "hello\r\nhello\r\r\n" Python 3.2: "hello\nhello\r\n" Python 3.3.0a4: "hello\nhello\r\n"
All on Windows.
<stifle o="self"/> Hm. Maybe it's that port's implementation of universal newlines or something like that? What happens if you use an explicit "end=" argument? (I don't have a Python 3 to check on Windows easily available.)
On 11 June 2012 07:16, Stephen J. Turnbull <stephen@xemacs.org> wrote:
MRAB writes:
> That's actually Python 3.1. From Python 3.2 it's slightly different, > but still not quite right: > > Python 3.1: "hello\r\nhello\r\r\n" > Python 3.2: "hello\nhello\r\n" > Python 3.3.0a4: "hello\nhello\r\n" > > All on Windows.
<stifle o="self"/>
Hm. Maybe it's that port's implementation of universal newlines or something like that? What happens if you use an explicit "end=" argument? (I don't have a Python 3 to check on Windows easily available.)
Explicit end= makes no difference to the behaviour. In fact, a minimal test suggests that universal newline mode is not enabled on Windows in Python 3. That's a regression from 2.x. See below. D:\Data>py -3 -c "print('x')" | od -c 0000000 x \n 0000002 D:\Data>py -2 -c "print('x')" | od -c 0000000 x \r \n 0000003 D:\Data>py -3 -V Python 3.2.2 D:\Data>py -2 -V Python 2.7.2 Paul.
2012/6/11 Paul Moore <p.f.moore@gmail.com>
Explicit end= makes no difference to the behaviour. In fact, a minimal test suggests that universal newline mode is not enabled on Windows in Python 3. That's a regression from 2.x. See below.
D:\Data>py -3 -c "print('x')" | od -c 0000000 x \n 0000002
D:\Data>py -2 -c "print('x')" | od -c 0000000 x \r \n 0000003
This is certainly related to http://bugs.python.org/issue11990 -- Amaury Forgeot d'Arc
On Mon, Jun 11, 2012 at 2:43 AM, Serhiy Storchaka <storchaka@gmail.com> wrote:
On 10.06.12 18:44, Nick Coghlan wrote:
This approach also has the advantage of leaving sys.__std(in/out/err)__ in a somewhat usable state.
And then sys.std* and sys.__std*__ have their own inconsistent buffers.
Correct, but using detach() leaves sys.__std*__ completely broken (either throwing exceptions or silently failing to emit output). Creating two independent streams that share the underlying file handle is much closer to the 2.x behaviour when replacing sys.std*. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Calling detach() on the standard streams is a bad idea - the interpreter uses the originals internally, and calling detach() breaks them. -- Sent from my phone, thus the relative brevity :) On Jun 10, 2012 6:03 AM, "Serhiy Storchaka" <storchaka@gmail.com> wrote:
On 09.06.12 12:55, Nick Coghlan wrote:
So, after much digging, it appears the *right* way to replace a standard stream in Python 3 after application start is to do the following:
sys.stdin = open(sys.stdin.fileno(), 'r',<new settings>) sys.stdout = open(sys.stdout.fileno(), 'w',<new settings>) sys.stderr = open(sys.stderr.fileno(), 'w',<new settings>)
sys.stdin = io.TextIOWrapper(sys.stdin.**detach(), <new settings>) sys.stdout = io.TextIOWrapper(sys.stdout.**detach(), <new settings>) ...
None of these methods are not guaranteed to work if the input or output have occurred before.
______________________________**_________________ Python-ideas mailing list Python-ideas@python.org http://mail.python.org/**mailman/listinfo/python-ideas<http://mail.python.org/mailman/listinfo/python-ideas>
Le 10/06/2012 04:26, Nick Coghlan a écrit :
Calling detach() on the standard streams is a bad idea - the interpreter uses the originals internally, and calling detach() breaks them.
Where does it do that? The interpreter certainly shouldn't hardwire the original objects internally. Moreover, your snippet is wrong because if someone replaces the streams for a second time, garbage collecting the previous streams will close the file descriptors. You should use closefd=False. Regards Antoine.
On Sun, Jun 10, 2012 at 5:17 PM, Antoine Pitrou <solipsis@pitrou.net> wrote:
Le 10/06/2012 04:26, Nick Coghlan a écrit :
Calling detach() on the standard streams is a bad idea - the interpreter uses the originals internally, and calling detach() breaks them.
Where does it do that? The interpreter certainly shouldn't hardwire the original objects internally.
At the very least, sys.__std(in/out/err)__. Doing "sys.stderr = io.TextIOWrapper(sys.stderr.detach(), line_buffering=True)" also seems to suppress display of exception tracebacks at the interactive prompt (perhaps the default except hook is using a cached reference?). I believe PyFatalError and other APIs that are used deep in the interpreter won't respect the module level setting. Basically, it's dangerous to use detach() on a stream where you don't hold the sole reference, and the safest approach with the standard streams is to assume that other code is holding references to them. Detaching the standard streams is just as likely to cause problems as closing them.
Moreover, your snippet is wrong because if someone replaces the streams for a second time, garbage collecting the previous streams will close the file descriptors. You should use closefd=False.
True, although that nicety is all the more reason to encapsulate this idiom in a new IOBase.reopen() method: def reopen(self, mode=None, buffering=-1, encoding=None, errors=None, newline=None, closefd=False): if mode is None: mode = getattr(mode, self, 'r') return open(self.fileno(), mode, buffering, encoding, errors, newline, closefd) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 10.06.12 05:26, Nick Coghlan wrote:
Calling detach() on the standard streams is a bad idea - the interpreter uses the originals internally, and calling detach() breaks them.
If interpreter uses standard streams then it uses raw C streams (FILE *) stdin/stdout/etc. Calling open(sys.stdin.fileno()) bypasses internal buffering in sys.stdin, sys.stdin.buffer, sys.stdin.decoder and raw C stdin (if it used in lower level), and lose and break multibyte characters.
sys.stdin = open(sys.stdin.fileno(), 'r',<new settings>) sys.stdout = open(sys.stdout.fileno(), 'w',<new settings>) sys.stderr = open(sys.stderr.fileno(), 'w',<new settings>)
sys.stdin = io.TextIOWrapper(sys.stdin.detach(), <new settings>) sys.stdout = io.TextIOWrapper(sys.stdout.detach(), <new settings>) ...
None of these methods are not guaranteed to work if the input or output have occurred before.
You should set the newline option for sys.std* files. Python 3 does something like this: if os.name == "win32: # translate "\r\n" to "\n" for sys.stdin on Windows newline = None else: newline = "\n" sys.stdin = io.TextIOWrapper(sys.stdin.detach(), newline=newline, <new settings>) sys.stdout = io.TextIOWrapper(sys.stdout.detach(), newline="\n", <new settings>) sys.stderr = io.TextIOWrapper(sys.stderr.detach(), newline="\n", <new settings>) -- Lib/test/regrtest.py uses the following code which is not exactly correct (it creates a new buffered writer instead of reusing sys.stdout buffered writer): def replace_stdout(): """Set stdout encoder error handler to backslashreplace (as stderr error handler) to avoid UnicodeEncodeError when printing a traceback""" import atexit stdout = sys.stdout sys.stdout = open(stdout.fileno(), 'w', encoding=stdout.encoding, errors="backslashreplace", closefd=False, newline='\n') def restore_stdout(): sys.stdout.close() sys.stdout = stdout atexit.register(restore_stdout) Victor
participants (9)
-
Amaury Forgeot d'Arc -
Antoine Pitrou -
Mark Lawrence -
MRAB -
Nick Coghlan -
Paul Moore -
Serhiy Storchaka -
Stephen J. Turnbull -
Victor Stinner