Python 3.0 automatic decoding of UTF16

Johannes Bauer dfnsonfsduifb at
Sat Dec 6 17:38:03 CET 2008

info at schrieb:

> 2 problems: endianness and trailing zer byte.
> This works for me:

This is very strange - when using "utf16", endianness should be detected
automatically. When I simply truncate the trailing zero byte, I receive:

Traceback (most recent call last):
  File "./", line 12, in <module>
    a = AddressBook("2008_11_05_Handy_Backup.txt")
  File "./", line 7, in __init__
    line = f.readline()
  File "/usr/local/lib/python3.0/", line 1807, in readline
    while self._read_chunk():
  File "/usr/local/lib/python3.0/", line 1556, in _read_chunk
    self._set_decoded_chars(self._decoder.decode(input_chunk, eof))
  File "/usr/local/lib/python3.0/", line 1293, in decode
    output = self.decoder.decode(input, final=final)
  File "/usr/local/lib/python3.0/", line 300, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
  File "/usr/local/lib/python3.0/encodings/", line 69, in
    return self.decoder(input, self.errors, final)
UnicodeDecodeError: 'utf16' codec can't decode byte 0x0a in position 0:
truncated data

But I suppose something *is* indeed weird because the file I uploaded
and which did not yield the "truncated data" error ia 1559 bytes, which
just cannot be.


"Meine Gegenklage gegen dich lautet dann auf bewusste Verlogenheit,
verlästerung von Gott, Bibel und mir und bewusster Blasphemie."
         -- Prophet und Visionär Hans Joss aka HJP in de.sci.physik
                         <48d8bf1d$0$7510$5402220f at>

More information about the Python-list mailing list