On Dec 21, 2007 12:05 AM, Nick Coghlan ncoghlan@gmail.com wrote:

Guido van Rossum wrote:

...

When I build from scratch and run most tests (regrtest.py -uall) I get some strange failures with test_sys.py:

test test_sys failed -- Traceback (most recent call last): File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py", line 302, in test_43581 self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding) AssertionError: 'ascii' != 'ISO-8859-1'

The same test doesn't fail when run in isolation.

Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6!

Any ideas?

It looks like the chunk of code in TextIOWrapper might be getting different answers when trying to work out the encoding for stdin and stderr (not that I can see how that would happen...). Or maybe there is some way that test_sys could be seeing the temporary '__stderr__' entry that is put in place until the io module is available?

What do you get for stdin/stdout/stderr.encoding at the interactive prompt? On Ubuntu, I get UTF-8 for all of them in both 3.0a2 and 2.5.1, but I'm not seeing the problem, either.

Other values of possible interest: os.device_encoding(1) os.device_encoding(2) locale.getpreferredencoding()

Another possibility would be to throw some debugging code into io.py to print out the encoding name to stderr when file is 1 or 2.

No answers yet, but a clue, and anyone on OSX should now be able to reproduce this (with 2.5, 2.6 or 3.0) as follows:

./python ./Lib/test/test_sys.py | cat

That is, the problem happens when stdout is redirected to a pipe (or a file).

-On [20080108 12:09], Nick Coghlan (ncoghlan@gmail.com) wrote:

Redirecting stdout also fails for both the trunk and the py3k branch for me on Ubuntu. If I redirected stderr as well then the tests worked again.

Given that a pipe/file and the console very likely *do* have different encodings, maybe the test is just wrong?

This sounds like a problem I recently blogged about (verbatim copy):

When you use Python with sys.stdout you might run into a problem where sys.stdout.encoding suddenly becomes None. This happens due to the fact that upon using a pipe or redirection, at least under Unix, it falls back to not knowing anything about the target. In order to work around this you can add a fallback to use locale.getpreferredencoding(). So if you use encode() on a string you can do something like:

from locale import getpreferredencoding

text = u"Something special"

print text.encode(sys.stdout.encoding or getpreferredencoding() or 'ascii', 'replace')

This is how we currently use it within Babel as well for printing the locale list.