[New-bugs-announce] [issue2614] Console UnicodeDecodeError s once more

[anatoly techtonik]

New submission from anatoly techtonik <techtonik at gmail.com>:

Python debugging under console is a PITA, because it has a bad habit to
fail with UnicodeEncodeError in case of unknown encoding in output. It
quickly turns into a headache when inspecting methods like in the
following example running under windows:

>>> import active_directory
>>> users = active_directory.search ("objectCategory='Person'",
"objectClass='User'")
>>> u = users.next()
>>> u = users.next()
>>> u = users.next()
>>> u.dump()
LDAP://CN=LыхъёрэфЁ,CN=Users,DC=dom,DC=com
{
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "build\bdist.win32\egg\active_directory.py", line 495, in dump
  File "C:\Python25\lib\codecs.py", line 303, in write
    data, consumed = self.encode(object, self.errors)
  File "C:\Python25\lib\encodings\cp1251.py", line 12, in en
code
    return codecs.charmap_encode(input,errors,encoding_table)
UnicodeDecodeError: 'ascii' codec can't decode byte 0x84 in position 8:
ordinal
not in range(128)

Will this be fixed in Py3k to allow range(255) in case of unknown
encoding? Or will there be a new wrapper function - some
rawhexprint(...) that will temporary change sys.output encoding for the
time arguments are executed and process out of range symbols in output
stream according to its embedded rules (i.e. converting to hex
representaton). Another function can be supplied to write raw bytes to
console as-is. Raw write is preferable as it gives the possibility to
redirect output to a file and inspect it later.

These encoding issues in my POV is the stumbling block that makes
scripting in Python 2.x harder for non-english users and hampers overall
Python adoption. Is this going change in Py3k?

----------
components: Unicode, Windows
messages: 65342
nosy: techtonik
severity: normal
status: open
title: Console UnicodeDecodeError s once more
versions: Python 2.5, Python 3.0

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2614>
__________________________________