[Tutor] UnicodeEncodeError

Kent Johnson kent37 at tds.net
Fri Oct 3 22:04:38 CEST 2008

On Fri, Oct 3, 2008 at 2:39 PM, Rob Sutherland <luckdancing at gmail.com> wrote:
> On Fri, Oct 3, 2008 at 12:23 PM, Kent Johnson <kent37 at tds.net> wrote:
>> It helps if you show the code that is causing the error and the full
>> traceback. Presumably you are calling someString.encode("some
>> encoding") where "some encoding"  is an encoding that doesn't i
> I'm encountering this error while attempting to store the string in
> a database, the code I execute is
> cur.execute(statement ,attachDict)
> which falls through to the code that actually does the encoding  in
> /usr/lib/python2.4/encodings/iso8859_1.py
> Traceback (most recent call last):
>  File "./mail2db.py", line 219, in ?
>    storemail(cur, conn,emailDict,attachment_hold)
>  File "./mail2db.py", line 118, in storemail
>    attachments_ok = storeattachment(cur,conn,search_message_id,attachment_hold)
>  File "./mail2db.py", line 61, in storeattachment
>    cur.execute(statement ,attachDict)
>  File "/usr/lib/python2.4/site-packages/psycopg2/extras.py", line 88,
> in execute
>    return _cursor.execute(self, query, vars, async)
>  File "/usr/lib/python2.4/encodings/iso8859_1.py", line 18, in encode
>    return codecs.charmap_encode(input,errors,encoding_map)
> UnicodeEncodeError: 'charmap' codec can't encode character u'\u2019'
> in position 144: character maps to <undefined>

My guess is, your database is set up to hold ISO-8859-1 data
(Latin-1). You are giving it Unicode data, so it tries to encode as
Latin-1. The data includes a character that is not in Latin-1 so it

I would try explicitly converting the data to latin-1 before you send
it to the database, giving it one of the forgiving error handling
methods I referred to earlier. Or, change your database to UTF-8...


PS Please use Reply All to reply to the list.

More information about the Tutor mailing list