Special chars with HTMLParser
stefan_ml at behnel.de
Fri Aug 7 08:53:38 CEST 2009
> I am parsing a web page with special chars such as é (which
> stands for é).
> I know I can have the unicode character é from unicode
> but with those extra characters I don' t know.
> I tried to implement handle_charref within HTMLParser without success.
> Furthermore, if I have the data abécd, handle_data will get "ab",
> handle_charref will get xe9 and then handle_data doesn't have the end
> of the string ("cd").
Any reason you can't use a tree based HTML parser like the one in
lxml.html? That would eliminate this kind of problem altogether, as you'd
always get a well-decoded unicode string from the tree content.
More information about the Python-list