UnicodeError: ASCII decoding error: ordinal not in range(128)

[Gerhard Haering]

In article <ah9huc$4eb$1 at newsreader.mailgate.org>, eugene kim wrote:
> hi..
> i'm trying to parse a xml file and feed into postgresql...
> the xml file contains korean..
> is this python problem or postgres problem?

An understanding problem. I don't need to read the rest of your message because
there's exactly one reason for the error message in the subject: you're trying
to convert a Unicode object into a string object and don't provide the encoding
to use. For example if you want to convert Unicode into an utf-8 encoded
string, you can do something like:

x = unicode("Ísterreich", "iso-8859-1") # create a Unicode object
print x.encode("utf-8") # convert it into a Python string

> sqlclause = 'INSERT INTO history VALUES (' + tmp + self.url + tmp +","+  
> tmp + self.title + tmp + "," + self.firsttime + "," + self.lasttime + "," + 
> self.visits + " )"

Also it's not advisable to create SQL strings yourself. It's much better to use
a DB-API module like pyPgSQL, psycopg (or even PyGreSQL's pgdb ;-) and let the
database module quote your parameters:

...
cursor.execute("insert into history (col1, col1, col3) values (%s, %s, %s)",
                 (self.url, self.title, self.firsttime))

This will use the proper quoting of the parameters (escape apostrophes, etc.).
For pyPgSQL, there's even a patch waiting for testers that will allow you to
use Unicode strings here.

For now you'll have to encode your Unicode strings that you get from parsing
the XML into Python strings of your favourite encoding.

> import _pg

I'd recommend to stay away from low-level modules with underscores in front.
PyGreSQL (if you want to use it) has a DB-API wrapper in the pgdb module.

Gerhard
-- 
mail:   gerhard <at> bigfoot <dot> de       registered Linux user #64239
web:    http://www.cs.fhm.edu/~ifw00065/    OpenPGP public key id 86AB43C0
public key fingerprint: DEC1 1D02 5743 1159 CD20  A4B6 7B22 6575 86AB 43C0
reduce(lambda x,y:x+y,map(lambda x:chr(ord(x)^42),tuple('zS^BED\nX_FOY\x0b')))