[I18n-sig] Codec Language

[Andy Robinson]

On the subject of a mini-language for dealing with Asian codecs...I'm
fooling around with something in pure Python - a toy interpreter for a basic
FSM - I'll try to post something up after the weekend.  In the meantime, we
should certainly list the actions we need to be able to perform at a
conceptual level:


1. Data structures/types for bytes, strings, numbers and mapping tables
2. Read n bytes into designated buffers from input
3. Write contents of designated buffers to output
4. Look up contents of a buffer in a mapping table, and do somethign with
the output (how to deal with failed lookups?)
5. Do math, string concenatenation, bit operations
6. Wide range of pattern-matching tests on short strings and bytes - byte in
range, byte in set etc.  mxTextTools gives loads of examples.

Please pitch in with any suggested operations you think we need.

The real issue seems to be, can we do it with an FSM that is not hideously
complex to program?  Or do we need a non-finite language in which infinite
loops etc. are possible?  The latter is easier to write things in, but may
not be as safe or as fast.

- Andy