Readlines returns non ASCII character
ian.g.kelly at gmail.com
Wed Sep 23 18:07:36 CEST 2015
On Wed, Sep 23, 2015 at 6:47 AM, SANKAR . <shankarphy at gmail.com> wrote:
> Hi all,
> I am not a expert programmer but I have to extract information from a large
> I used codecs.open(..) with UTF16 encoding to read this file. It could
> read all the lines in the file but returns with the non Ascii characters.
> Below are 5 sample lines. How do I avoid having this non Ascii items. Is
> there a better way to read this?
I suspect that what you want is not "non-ASCII" but just to read the
file without all the mojibake, which is likely an indication that
you're using the wrong encoding.
Do you know that UTF-16 is actually the encoding of the file?
Based on the spaces that appear between adjacent characters, I would
guess that this is probably in a 32-bit encoding, perhaps UTF-32. On
the other hand, the repeated 0x00ff 0x00fe 0x00ff are very curious; I
don't see how that could be valid UTF-32. Are you sure that this is a
text file and not some propietary binary data format?
More information about the Python-list