Re: [Python-Dev] Support of UTF-16 and UTF-32 source encodings

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

I don't see any good reason for allowing non-ASCII-compatible encodings in the reference CPython interpreter.

There might be a case for having the tokenizer not care about encodings at all and just operate on a stream of unicode characters provided by a different layer.