[I18n-sig] PEP 263 and Japanese native encodings

[M.-A. Lemburg]

"Martin v. Loewis" wrote:
> 
> "M.-A. Lemburg" <mal@lemburg.com> writes:
> 
> > Martin's patch leaves these "minor" issues to the tokenizer
> > and that's good :-)
> >
> > I only wanted to give a very simple
> > example of what the original idea was when I added "ASCII
> > compatible encoding" to the PEP -- basically to simplify
> > the coding parsing part.
> 
> In my implementation, the "ASCII superset" restriction is stronger,
> though: the tokenizer needs to find the end of a string without
> decoding it. That is not possible for some of the encodings that pass
> your "ASCII superset" test.

I hope I have clarified the intent of the definition that was
in the PEP -- I've removed it a few days ago and replaced it
with a hopefully better description of what was meant. Your
implementation is just fine in this respect (at least from looking
at it ;-).

Are there any other common encodings which could have trouble 
with the implementation besides two or more byte encodings ?

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Company & Consulting:                           http://www.egenix.com/
Python Software:                   http://www.egenix.com/files/python/