[Python-Dev] Import and unicode: part two

[James Y Knight]

On Jan 19, 2011, at 6:44 PM, Toshio Kuratomi wrote:
> This problem of which encoding to use is a problem that can be
> seen on UNIX systems even now.  Try this:
> 
>  echo 'print("hi")' > café.py
>  convmv -f utf-8 -t latin1 café.py
>  python3 -c 'import café'
> 
> ASCII seems very sensible to me when faced with these ambiguities.
> 
> Other options I can brainstorm that could be explored:
> 
> * Specify an encoding per platform and stick to that.  (So, for instance,
>  all module names on posix platforms would have to be utf-8).  Force
>  translation between encoding when installing packages (But that doesn't
>  help for people that are creating their modules using their own build
>  scripts rather than distutils, copying the files using raw tar, etc.)
> * Change import semantics to allow specifying the encoding of the module on
>  the filesystem (seems really icky).

None of this is unique to import -- the same exact issue occurs with open(u'café'). I don't see any reason why import café should be though of as more of a problem, or treated any differently.

It's reasonable to recommend that people use ASCII in their module names if they want wide portability, but it should still be supported to use non-ASCII.

James