UnicodeEncodeError when not running script from IDE

Magnus Pettersson magpettersson at gmail.com
Tue Feb 12 18:12:42 CET 2013

> What encoding is this file?  Since you're appending to it, you really 
> need to match the pre-existing encoding, or the next program to deal 
> with it is in big trouble.  So using the io.open() without the encoding= 
> keyword is probably a mistake.

The .txt file is in UTF-8

I have got it to work now in the terminal, but i dont understand what im doing and why i didnt need to do all the unicode strings and encode mumbo jumbo in eclipse

#Here kanji = u"私"
baseurl = u"http://www.romajidesu.com/kanji/"
url = baseurl+kanji
savefile([url]) #this test works now. uses: io.open(filepath, "a",encoding="UTF-8") as f:
# This made the fetching of the website work. Why did i have to write url.encode("UTF-8") when url already is unicode? I feel i dont have a good understanding of this.
page = urllib2.urlopen(url.encode("UTF-8"))


More information about the Python-list mailing list