unicode + xml

[Mark Tolonen]

"Laurent Luce" <laurentluce49 at yahoo.com> wrote in message 
news:255473.44957.qm at web54203.mail.re2.yahoo.com...
> Hello,
>
> I am trying to do the following:
>
> - read list of folders in a specific directory: os.listdir() - some 
> folders have Japanese characters
> - post list of folders as xml to a web server: I used content-type 
> 'text/xml' and I use '<?xml version="1.0" encoding="utf-8"?>' to start the 
> xml data.
> - on the server side (Django), I get the data using post_data and I use 
> minidom.parseString() to parse it. I get an exception because of the 
> following in the xml for one of the folder name:
> '/ufffdX/ufffd^/ufffd[/ufffdg /ufffd/ufffd/ufffdj/ufffd/ufffd/ufffd['
>
> The weird thing is that I see 5 bytes for each unicode character: ie: 
> /ufffdX
>
> Should I format the data differently inside the xml so minidom is happy ?

You aren't seeing 5 bytes for each unicode character.  You are seeing 
'\ufffd' (the code point REPLACEMENT_CHARACTER) intermixed with other 
characters.  The wrong encoding was probably used to decode the filename 
byte strings to Unicode.

We can give more specific help if you specify your operating system and 
version of Python used.

-Mark