htmllib.HTMLParser and unicode

[Achim Domma]

Hi,

should the HTMLParser be able to handle unicode input? I get the following
traceback:

    self.feed(self.data)
  File "C:\Python23\lib\sgmllib.py", line 94, in feed
    self.goahead(0)
  File "C:\Python23\lib\sgmllib.py", line 183, in goahead
    self.handle_entityref(name)
  File "C:\Python23\lib\sgmllib.py", line 390, in handle_entityref
    self.handle_data(table[name])
  File "C:\Python23\lib\htmllib.py", line 49, in handle_data
    self.savedata = self.savedata + data
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa0 in position 0:
ordinal not in range(128)

The input is a html page from the web, encoded as utf8. I converted the
string via data.decode('utf8'). The result is passed to the feed function.

regards,
Achim