FYI, I've uploaded a new version of the proposal which includes new codec APIs, a new codec search mechanism and some minor fixes here and there.

The latest version of the proposal is available at:

http://starship.skyport.net/~lemburg/unicode-proposal.txt

Older versions are available as:

http://starship.skyport.net/~lemburg/unicode-proposal-X.X.txt

Some POD (points of discussion) that are still open:

· Unicode objects support for %-formatting

· Design of the internal C API and the Python API for the Unicode character properties database

Skip Montanaro wrote:

I haven't been following this discussion closely at all, and have no previous experience with Unicode, so please pardon a couple stupid questions from the peanut gallery:

1. What does U+0061 mean (other than 'a')?  That is, what is U?

U+XXXX means Unicode character with ordinal hex number XXXX. It is basically just another way to say, hey I want the Unicode character at position 0xXXXX in the Unicode spec.

2. I saw nothing about encodings in the Codec/StreamReader/StreamWriter
   description.  Given a Unicode object with encoding e1, how do I write
   it to a file that is to be encoded with encoding e2?  Seems like I
   would do something like

       u1 = unicode(s, encoding=e1)
       f = open("somefile", "wb")
       u2 = unicode(u1, encoding=e2)
       f.write(u2)

   Is that how it would be done?  Does this question even make sense?

The unicode() constructor converts all input to Unicode as basis for other conversions. In the above example, s would be converted to Unicode using the assumption that the bytes in s represent characters encoded using the encoding given in e1. The line with u2 would raise a TypeError, because u1 is not a string. To convert a Unicode object u1 to another encoding, you would have to call the .encode() method with the intended new encoding. The Unicode object will then take care of the conversion of its internal Unicode data into a string using the given encoding, e.g. you'd write:

f.write(u1.encode(e2))

3. What will the impact be on programmers such as myself currently
   living with blinders on (that is, writing in plain old 7-bit ASCII)?

If you don't want your scripts to know about Unicode, nothing will really change. In case you do use e.g. Latin-1 characters in your scripts for strings, you are asked to include a pragma in the comment lines at the beginning of the script (so that programmers viewing your code using other encoding have a chance to figure out what you've written).

Here's the text from the proposal: """ Note that you should provide some hint to the encoding you used to write your programs as pragma line in one the first few comment lines of the source file (e.g. '# source file encoding: latin-1'). If you only use 7-bit ASCII then everything is fine and no such notice is needed, but if you include Latin-1 characters not defined in ASCII, it may well be worthwhile including a hint since people in other countries will want to be able to read you source strings too. """

Other than that you can continue to use normal strings like you always have.

Hope that clarifies things at least a bit,

Pretty quiet around here lately...

My guess is that most positions and opinions have been covered. It is now probably time for less talk, and more code!

It is time to start an implementation plan? Do we start with /F's Unicode implementation (which /G *smirk* seemed to approve of)? Who does what? When can we start to play with it?

And a key point that seems to have been thrust in our faces at the start and hardly mentioned recently - does the proposal as it stands meet our sponsor's (HP) requirements?

Mark.