[spambayes-bugs] [ spambayes-Bugs-1187208 ] import into CDB chokes on 8-bit chars

[SourceForge.net]

Bugs item #1187208, was opened at 2005-04-21 20:45
Message generated for change (Comment added) made by anadelonbrin
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1187208&group_id=61702

>Category: None
Group: 1.1.x
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: Leonid (leobru)
>Assigned to: Tony Meyer (anadelonbrin)
Summary: import into CDB chokes on 8-bit chars

Initial Comment:
If  the CSV file contains an iso-8859-1 character,
import into CDB fails:

file csv (2 lines, save as iso-8859-1):
1,1
fiancée,1,1

sb_dbexpimp.py -i -o
Storage:persistent_use_database:cdb -o
Storage:persistent_storage_file:cdb -v -f csv
Importing file csv into database /.../cdb
Storing database, please be patient.  Even moderately sized
databases may take a very long time to store.
Traceback (most recent call last):
  File "./sb_dbexpimp.py", line 248, in ?
    runImport(dbFN, useDBM, newDBM, flatFN)
  File "./sb_dbexpimp.py", line 200, in runImport
    bayes.store()
  File
"/usr/home/leob/spambayes-1.1a1/scripts/spambayes/storage.py",
line 649, in store
    cdb.cdb_make(db, items)
  File
"/usr/home/leob/spambayes-1.1a1/scripts/spambayes/cdb.py",
line 166, in cdb_make
    outfile.write(key)
UnicodeEncodeError: 'ascii' codec can't encode
character u'\xe9' in position 5: ordinal not in range(128)


----------------------------------------------------------------------

>Comment By: Tony Meyer (anadelonbrin)
Date: 2005-04-22 16:08

Message:
Logged In: YES 
user_id=552329

Thanks.  Fixed in storage.py r1.51

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=498103&aid=1187208&group_id=61702