Some questions about decode/encode
sjmachin at lexicon.net
Thu Jan 24 10:51:56 CET 2008
On Jan 24, 2:49 pm, glacier <rong.x... at gmail.com> wrote:
> I use chinese charactors as an example here.
> My first question is : what strategy does 'decode' use to tell the way
> to seperate the words. I mean since s1 is an multi-bytes-char string,
> how did it determine to seperate the string every 2bytes or 1byte?
The usual strategy for encodings like GBK is:
1. If the current byte is less than 0x80, then it's a 1-byte
2. Current byte 0x81 to 0xFE inclusive: current byte and the next byte
make up a two-byte character.
3. Current byte 0x80: undefined (or used e.g. in cp936 for the 1-byte
4: Current byte 0xFF: undefined
More information about the Python-list